Method and decoder for decoding a video bitstream using information in an SEI message

ABSTRACT

An electronic device for sending a message is described. The electronic device includes a processor and instructions stored in memory that is in electronic communication with the processor. The electronic device determines whether to include a common decoding unit CPB removal delay parameter in a picture timing Supplemental Enhancement Information (SEI) message. The electronic device also generates either a common decoding unit CPB removal delay parameter or a separate decoding unit CPB removal delay parameter for each decoding unit in the access unit. The electronic device also sends the picture timing SET message with the common decoding unit CPB removal delay parameter or the decoding unit CPB removal delay parameters.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a Divisional of copending application Ser. No.14/411,822, filed on Dec. 29, 2014, which is a the National Phase ofPCT/JP2013/004201 filed on Jul. 5, 2013, which claims priority under 35U.S.C. 119(e) to U.S. Provisional Application No. 61/668,921 filed onJul. 6, 2012, to U.S. Provisional Application No. 61/700,255 filed onSep. 12, 2012 and to U.S. Provisional Application No. 61/749,834 filedon Jan. 7, 2013, all of which are hereby expressly incorporated byreference into the present application.

TECHNICAL FIELD

The present disclosure relates generally to electronic devices. Morespecifically, the present disclosure relates to electronic devices forsignaling sub-picture based hypothetical reference decoder parameters.

BACKGROUND ART

Electronic devices have become smaller and more powerful in order tomeet consumer needs and to improve portability and convenience.Consumers have become dependent upon electronic devices and have come toexpect increased functionality. Some examples of electronic devicesinclude desktop computers, laptop computers, cellular phones, smartphones, media players, integrated circuits, etc.

Some electronic devices are used for processing and displaying digitalmedia. For example, portable electronic devices now allow for digitalmedia to be consumed at almost any location where a consumer may be.Furthermore, some electronic devices may provide download or streamingof digital media content for the use and enjoyment of a consumer.

SUMMARY OF INVENTION Technical Problem

The increasing popularity of digital media has presented severalproblems. For example, efficiently representing high-quality digitalmedia for storage, transmittal and rapid playback presents severalchallenges. As can be observed from this discussion, systems and methodsthat represent digital media efficiently with improved performance maybe beneficial.

Solution to Problem

According to the present invention, there is provided an electronicdevice for buffering a bitstream, comprising: determining a picturetiming SEI (Supplemental Enhancement Information) message to removedata; determining an access unit CPB (Coded Picture Buffer) removaldelay parameter for removing an access unit from the CPB in the picturetiming SEI message; and determining one or more decoding unit CPBremoval delays for each decoding unit in the access unit; wherein thesum of the decoding unit CPB removal delays is equal to the access unitCPB removal delay parameter.

According to the present invention, there is provided an electronicdevice for sending a message, comprising: a processor; memory inelectronic communication with the processor; instructions stored in thememory, the instructions being executable to: determine, when a CodedPicture Buffer (CPB) supports operation on a sub-picture level, whetherto include a common decoding unit CPB removal delay parameter in apicture timing Supplemental Enhancement Information (SEI) message;generate, when the common decoding unit CPB removal delay parameter isto be included in the picture timing SEI message, the common decodingunit CPB removal delay parameter, wherein the common decoding unit CPBremoval delay parameter is applicable to all decoding units in an accessunit from the CPB; generate, when the common decoding unit CPB removaldelay parameter is not to be included in the picture timing SEI message,a separate decoding unit CPB removal delay parameter for each decodingunit in the access unit; and send the picture timing SEI message withthe common decoding unit CPB removal delay parameter or the decodingunit CPB removal delay parameters.

According to the present invention, there is provided an electronicdevice for buffering a bitstream, comprising: a processor; memory inelectronic communication with the processor; instructions stored in thememory, the instructions being executable to: determine that a CPBsignals parameters on a sub-picture level for an access unit; determine,when a received picture timing Supplemental Enhancement Information(SEI) message comprises the common decoding unit Coded Picture Buffer(CPB) removal delay flag, a common decoding unit CPB removal delayparameter applicable to all decoding units in the access unit;determine, when the picture timing SEI message does not comprise thecommon decoding unit CPB removal delay flag, a separate decoding unitCPB removal delay parameter for each decoding unit in the access unit;remove decoding units from the CPB using the common decoding unit CPBremoval delay parameter or the separate decoding unit CPB removal delayparameters; and decode the decoding units in the access unit.

According to the present invention, there is provided a method forsending a message, comprising: determining, when a Coded Picture Buffer(CPB) supports operation on a sub-picture level, whether to include acommon decoding unit CPB removal delay parameter in a picture timingSupplemental Enhancement Information (SEI) message; generating, when thecommon decoding unit CPB removal delay parameter is to be included inthe picture timing SET message, the common decoding unit CPB removaldelay parameter, wherein the common decoding unit CPB removal delayparameter is applicable to all decoding units in an access unit from theCPB; generating, when the common decoding unit CPB removal delayparameter is not to be included in the picture timing SEI message, aseparate decoding unit CPB removal delay parameter for each decodingunit in the access unit; and sending the picture timing SET message withthe common decoding unit CPB removal delay parameter or the decodingunit CPB removal delay parameters.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example of one or moreelectronic devices in which systems and methods for sending a messageand buffering a bitstream may be implemented;

FIG. 2 is a flow diagram illustrating one configuration of a method forsending a message;

FIG. 3 is a flow diagram illustrating one configuration of a method fordetermining one or more removal delays for decoding units in an accessunit;

FIG. 4 is a flow diagram illustrating one configuration of a method forbuffering a bitstream;

FIG. 5 is a flow diagram illustrating one configuration of a method fordetermining one or more removal delays for decoding units in an accessunit;

FIG. 6 is a block diagram illustrating one configuration of an encoder604 on an electronic device;

FIG. 7 is a block diagram illustrating one configuration of a decoder onan electronic device;

FIG. 8 illustrates various components that may be utilized in atransmitting electronic device;

FIG. 9 is a block diagram illustrating various components that may beutilized in a receiving electronic device;

FIG. 10 is a block diagram illustrating one configuration of anelectronic device in which systems and methods for sending a message maybe implemented; and

FIG. 11 is a block diagram illustrating one configuration of anelectronic device in which systems and methods for buffering a bitstreammay be implemented.

DESCRIPTION OF EMBODIMENTS

An electronic device for sending a message is described. The electronicdevice includes a processor and instructions stored in memory that is inelectronic communication with the processor. The electronic devicedetermines, when a Coded Picture Buffer (CPB) supports operation on asub-picture level, whether to include a common decoding unit CPB removaldelay parameter in a picture timing Supplemental Enhancement Information(SEI) message. The electronic device also generates, when the commondecoding unit CPB removal delay parameter is to be included in thepicture timing SEI message (or some other SEI message or some otherparameter set e.g. picture parameter set or sequence parameter set orvideo parameter set or adaptation parameter set), the common decodingunit CPB removal delay parameter, wherein the common decoding unit CPBremoval delay parameter is applicable to all decoding units in an accessunit from the CPB. The electronic device also generates, when the commondecoding unit CPB removal delay parameter is not to be included in thepicture timing SEI message, a separate decoding unit CPB removal delayparameter for each decoding unit in the access unit. The electronicdevice also sends the picture timing SET message with the commondecoding unit CPB removal delay parameter or the decoding unit CPBremoval delay parameters.

The common decoding unit CPB removal delay parameter may specify anamount of sub-picture clock ticks to wait after removal from the CPB ofa immediately preceding decoding unit before removing from the CPB acurrent decoding unit in the access unit associated with the picturetiming SEI message.

Furthermore, when a decoding unit is a first decoding unit in an accessunit, the common decoding unit CPB removal delay parameter may specifyan amount of sub-picture clock ticks to wait after removal from the CPBof a last decoding unit in an access unit associated with a most recentbuffering period SEI message in a preceding access unit before removingfrom the CPB the first decoding unit in the access unit associated withthe picture timing SEI message.

In contrast, when the decoding unit is a non-first decoding unit in anaccess unit, the common decoding unit CPB removal delay parameter mayspecify an amount of sub-picture clock ticks to wait after removal fromthe CPB of a preceding decoding unit in the access unit associated withthe picture timing SET message before removing from the CPB a currentdecoding unit in the access unit associated with the picture timing SEImessage.

The decoding unit CPB removal delay parameters may specify an amount ofsub-picture clock ticks to wait after removal from the CPB of the lastdecoding unit before removing from the CPB an i-th decoding unit in theaccess unit associated with the picture timing SEI message.

The electronic device may calculate the decoding unit CPB removal delayparameters according to a remainder of a modulo 2^((cpb) ^(_) ^(removal)^(_) ^(delay) ^(_) ^(length) ^(_) ^(minus1+1)) counter wherecpb_removal_delay_length_minus1+1 is a length of a common decoding unitCPB removal delay parameter.

The electronic device may also generate, when the CPB supports operationon an access unit level, a picture timing SET message including a CPBremoval delay parameter that specifies how many clock ticks to waitafter removal from the CPB of an access unit associated with a mostrecent buffering period SEI message in a preceding access unit beforeremoving from the CPB the access unit data associated with the picturetiming SET message.

The electronic device may also determine whether the CPB supportsoperation on a sub-picture level or an access unit level. This mayinclude determining a picture timing flag that indicates whether a CodedPicture Buffer (CPB) provides parameters supporting operation on asub-picture level based on a value of the picture timing flag. Thepicture timing flag may be included in the picture timing SEI message.

Determining whether to include a common decoding unit CPB removal delayparameter may include setting a common decoding unit CPB removal delayflag to 1 when the common decoding unit CPB removal delay parameter isto be included in the picture timing SEI message. It may also includesetting the common decoding unit CPB removal delay flag to 0 when thecommon decoding unit CPB removal delay parameter is not to be includedin the picture timing SEI message. The common decoding unit CPB removaldelay flag may be included in the picture timing SEI message.

The electronic device may also generate, when the CPB supports operationon a sub-picture level, separate network abstraction layer (NAL) unitsrelated parameters that indicate an amount, offset by one, of NAL unitsfor each decoding unit in an access unit. Alternatively, or in additionto, the electronic device may generate a common NAL parameter thatindicates an amount, offset by one, of NAL units common to each decodingunit in an access unit.

An electronic device for buffering a bitstream is also described. Theelectronic device includes a processor and instructions stored in memorythat is in electronic communication with the processor. The electronicdevice determines that a CPB signals parameters on a sub-picture levelfor an access unit. The electronic device also determines, when areceived picture timing Supplemental Enhancement Information (SEI)message comprises the common decoding unit Coded Picture Buffer (CPB)removal delay flag, a common decoding unit CPB removal delay parameterapplicable to all decoding units in the access unit. The electronicdevice also determines, when the picture timing SEI message does notcomprise the common decoding unit CPB removal delay flag, a separatedecoding unit CPB removal delay parameter for each decoding unit in theaccess unit. The electronic device also removes decoding units from theCPB using the common decoding unit CPB removal delay parameter or theseparate decoding unit CPB removal delay parameters. The electronicdevice also decodes the decoding units in the access unit.

In one configuration, the electronic device determines that a picturetiming flag is set in the picture timing SEI message. The electronicdevice may also set a CPB removal delay parameter, cpb_removal_delay,according to

$\begin{matrix}{{{cpb\_ removal}{\_ delay}} = \frac{\begin{matrix}{\left( {\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_{minus}1}}{{du\_ cpb}{\_ removal}{{\_ delay}\lbrack i\rbrack}}} \right)*} \\t_{c,{sub}}\end{matrix}}{t_{c}}} & \left\lbrack {{Math}.\mspace{14mu} 1} \right\rbrack\end{matrix}$

where du_cpb_removal_delay[i] are the decoding unit CPB removal delayparameters, t_(c) is a clock tick, t_(c,sub) is a sub-picture clocktick, num_decoding_units_minus1 is an amount of decoding units in theaccess unit offset by one, and i is an index.

Alternatively, the electronic device may set a CPB removal delayparameter, cpb_removal_delay, anddu_cpb_removal_delay[num_decoding_units_minus1] so as to satisfy theequation

$\begin{matrix}{{- 1} \leq \left\lbrack {{{cpb\_ removal}{\_ delay}*t_{c}} - \left( {\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_ minus}\; 1}{{du\_ cpb}{\_ removal}{{\_ delay}\lbrack i\rbrack}*t_{c,{sub}}}} \right)} \right\rbrack \leq 1} & \left\lbrack {{Math}.\mspace{14mu} 2} \right\rbrack\end{matrix}$

where du_cpb_removal_delay[i] are the decoding unit CPB removal delayparameters, t_(c) is a clock tick, t_(c,sub) is a sub-picture clocktick, num_decoding_units_minus1 is an amount of decoding units in theaccess unit offset by one, and i is an index.

Alternatively, the electronic device may set a CPB removal delayparameter, cpb_removal_delay, and du_cpb_removal_delay[num_decodingunits_minus1] according tocpb_removal_delay*t_(c)=du_cpb_removal_delay[num_decoding_units_minus1]*t_(c)_(_) _(sub) where du_cpb_removal_delay[num_decoding_units_minus1] is thedecoding unit CPB removal delay parameter for thenum_decoding_units_minus1′th decoding unit, t_(c) is a clock tick,t_(c,sub) is a sub-picture clock tick num_decoding_units_minus1 is anamount of decoding units in the access unit offset by one.

In one configuration, the electronic device determines that a picturetiming flag is set in the picture timing SEI message. The electronicdevice may also set CPB removal delay parameters, cpb_removal_delay, anddu_cpb_removal_delay[num_decoding_units_minus1] so as to satisfy theequation:−1<=(cpb_removal_delay*t_(c)−du_cpb_removal_delay[num_decoding_units_minus1]*t_(c,sub))<=1where du_cpb_removal_delay[num_decoding_units_minus1] is the decodingunit CPB removal delay parameter for the num_decoding_units_minus1′thdecoding unit, t_(c) is a clock tick, t_(c,sub) is a sub-picture clocktick num_decoding_units_minus1 is an amount of decoding units in theaccess unit offset by one.

A ClockDiff variable may be defined asClockDiff=(num_units_in_tick<(num_units_in_sub_tick*(num_decoding_units_minus1+1))/time_scale)where num_units_in_tick is number of time units of a clock operating atthe frequency time_scale Hz that corresponds to one increment of a clocktick counter, num_units_in_sub_tick is number of time units of a clockoperating at the frequency time_scale Hz that corresponds to oneincrement of a sub-picture clock tick counter,num_decoding_units_minus1+1 is an amount of decoding units in the accessunit, and time_scale is the number of time units that pass in onesecond.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1, the CPB is operating at sub-picture level andClockDiff is greater than zero, the removal time for decoding unit m,t_(r)(m) is determined according to: t_(r)(m)=t_(r,n)(m)+t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub)) ClockDiff wheret_(r,n)(m) is the nominal removal time of the decoding unit m, t_(c)_(_) _(sub) is a sub-picture clock tick, Ceil( ) is a ceiling functionand t_(af)(m) is final arrival time of decoding unit m.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(n)<t_(a,f)(n), a picture timingflag is set to 1, the CPB is operating at an access unit level andClockDiff is greater than zero, the removal time for access unit n,t_(r)(n) is determined according to:t_(r)(n)=t_(r,n)(n)+t_(c)*Ceil((t_(af)(n)−t_(r,n)(n)))/t_(c))−ClockDiff,where t_(r,n)(n) is the nominal removal time of the access unit n, t_(c)is a clock tick, Ceil( ) is a ceiling function and t_(af)(n) is a finalarrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the last decoding unit m of access unit, t_(r)(m)according to: t_(r)(m)=t_(r,n)(m)+max((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/tc))) where t_(r,n)(m) is the nominalremoval time of the last decoding unit m, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,t_(r)(n) according to: t_(r)(n)=t_(r,n)(n)+max((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c)))) where t_(r,n)(m) is thenominal removal time of the last decoding unit n, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the last decoding unit m of access unit, t_(r)(m)according to: t_(r)(m)=t_(r,n)(m)+min((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c)))) where t_(r,n)(m) is thenominal removal time of the last decoding unit m, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,t_(r)(n) according to: t_(r)(n)=t_(r,n)(n)+min((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c)))) where t_(r,n)(m) is thenominal removal time of the last decoding unit n, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the last decoding unit m of access unit, tr(m)according to:t_(r)(m)=t_(r,n)(m)(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit m,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick and t_(af)(n) is a final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,tr(n) according to:t_(r)(n)=t_(r,n)(n)+(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit n,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick and t_(af)(n) is a final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for a decoding unit m which is not the last decoding unitis set as t_(r)(m)=t_(af)(m), where t_(af)(m) is a final arrival time ofdecoding unit m. When a low delay hypothetical reference decoder (HRD)flag (e.g., low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), apicture timing flag is set to 1 and the CPB is operating at sub-picturelevel, the removal time for a decoding unit m which is the last decodingunit m of access unit, t_(r)(m) according to: t_(r)(m)=t_(r,n)(m)+(t_(c)_(_) _(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit m,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( )i s a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick, t_(af)(n) is a final arrival time of access unit n, andt_(af)(m) is a final arrival time of last decoding unit m in the accessunit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for a decoding unit m which is not the last decoding unitis set as t_(r)(m)=t_(af)(m), where t_(af)(m) is a final arrival time ofdecoding unit m. When a low delay hypothetical reference decoder (HRD)flag (e.g., low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), apicture timing flag is set to 1 and the CPB is operating at sub-picturelevel, the removal time for a decoding unit m which is the last decodingunit m of access unit, t_(r)(m) according to:t_(r)(m)=t_(r,n)(m)+(t_(c)*Ceil((t_(af))−t_(r,n)(m))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit in,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick, t_(af)(n) is a final arrival time of access unit n, andt_(af)(m) is a final arrival time of last decoding unit m in the accessunit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for a decoding unit m is set as t_(r)(m)=t_(af)(m) wheret_(r,n)(m) is the nominal removal time of the decoding unit m, t_(c)_(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceiling function,t_(af)(m) is a final arrival time of decoding unit m, t_(r,n)(n) is thenominal removal time of the access unit n, t_(c) is clock tick,t_(af)(n) is a final arrival time of access unit n, and t_(af)(m) is afinal arrival time of decoding unit m in the access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,t_(r)(n) according to: t_(r)(n)=t_(af)(n) where t_(r,n)(m) is thenominal removal time of the last decoding unit n, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

Additionally in some cases a flag may be sent in part of the bitstreamto signal which of the above alternative equations are used for decidingthe removal time of the decoding units and removel time of the accessunit. In one case the flag may be called du_au_cpb_alignment_mode_flag.If du_au_cpb_alignment_mode_flag is 1 then the equations above whichalign the operation of CPB which operates in sub-picture based mode withthe CPB which operates in the access unit mode are used. Ifdu_au_cpb_alignment_mode_flag is 0 then the equations above which do notalign the operation of CPB which operates in sub-picture based mode withthe CPB which operates in the access unit mode are used.

In once case the flag du_au_cpb_alignment_mode_flag may be signaled inthe video usbility information (VUI). In another case the flagdu_au_cpb_alignment_mode_flag may be sent in picture timing SEI message.In yet another case the flag du_au_cpb_alignment_mode_flag may be sentin some other normative part of the bitstream. One example of modifiedsyntax and semantics in accordance with the systems and methodsdisclosed herein is given in Table (0) as follows.

TABLE 0 pic_timing( payloadSize ) {  if ( CpbDpbDelaysPresentFlag {  cpb_removal_delay   dpb_output_delay   if (sub_pic_cpb_params_present_flag ) {    num_decoding_units_minus1   du_au_cpb_alignment_mode_flag    for ( i = 0; i <=num_decoding_units_minus1; i++) {     num_nalus_in_du_minus1[ i ]    du_cpb_removal_delay[ i ]    }   }  } }

It should be noted that different symbols (names) than those used abovefor various variables may be used. For example t_(r)(n) of access unit nmay be called CpbRemovalTime(n), t_(r)(m) of decoding unit n may becalled CpbRemovalTime(m), t_(c) _(_) _(sub) may be called ClockSubTick,t_(c) may be called ClockTick, t_(af)(n) of access unit m may be calledFinalArrivalTime(n) of access unit n, t_(af)(m) of decoding unit m maybe called FinalArrivalTime(m), t_(r,n)(n) may be calledNominalRemovalTime(n) of the access unit n, t_(r,n)(m) may be calledNominalRemovalTime(m) of the decoding unit m.

A method for sending a message by an electronic device is alsodescribed. The method includes determining, when a Coded Picture Buffer(CPB) supports operation on a sub-picture level, whether to include acommon decoding unit CPB removal delay parameter in a picture timingSupplemental Enhancement Information (SEI) message. The method alsoincludes generating, when the common decoding unit CPB removal delayparameter is to be included in the picture timing SEI message, thecommon decoding unit CPB removal delay parameter, wherein the commondecoding unit CPB removal delay parameter is applicable to all decodingunits in an access unit from the CPB. The method also includesgenerating, when the common decoding unit CPB removal delay parameter isnot to be included in the picture timing SEI message, a separatedecoding unit CPB removal delay parameter for each decoding unit in theaccess unit. The method also includes sending the picture timing SEImessage with the common decoding unit CPB removal delay parameter or thedecoding unit CPB removal delay parameters.

A method for buffering a bitstream by an electronic device is alsodescribed. The method includes determining that a CPB signals parameterson a sub-picture level for an access unit. The method also includesdetermining, when a received picture timing Supplemental EnhancementInformation (SEI) message comprises the common decoding unit CodedPicture Buffer (CPB) removal delay flag, a common decoding unit CPBremoval delay parameter applicable to all decoding units in the accessunit. The method also includes determining, when the picture timing SEImessage does not comprise the common decoding unit CPB removal delayflag, a separate decoding unit CPB removal delay parameter for eachdecoding unit in the access unit. The method also includes removingdecoding units from the CPB using the common decoding unit CPB removaldelay parameter or the separate decoding unit CPB removal delayparameters. The method also includes decoding the decoding units in theaccess unit.

The systems and methods disclosed herein describe electronic devices forsending a message and buffering a bitstream. For example, the systemsand methods disclosed herein describe buffering for bitstreams startingwith sub-picture parameters. In some configurations, the systems andmethods disclosed herein may describe signaling sub-picture basedHypothetical Reference Decoder (HRD) parameters. For instance, thesystems and methods disclosed herein describe modification to a picturetiming Supplemental Enhancement Information (SEI) message. The systemsand methods disclosed herein (e.g., the HRD modification) may result inmore compact signaling of parameters when each sub-picture arrives andis removed from CPB at regular intervals.

Furthermore, when the sub-picture level CPB removal delay parameters arepresent, the Coded Picture Buffer (CPB) may operate at access unit levelor sub-picture level. The present systems and methods may also impose abitstream constraint so that the sub-picture level based CPB operationand the access unit level CPB operation result in the same timing ofdecoding unit removal. Specifically the timing of removal of lastdecoding unit in an access unit when operating in sub-picture mode andthe timing of removal of access unit when operating in access unit modewill be the same.

It should be noted that although the term “hypothetical” is used inreference to an HRD, the HRD may be physically implemented. For example,“HRD” may be used to describe an implementation of an actual decoder. Insome configurations, an HRD may be implemented in order to determinewhether a bitstream conforms to High Efficiency Video Coding (HEVC)specifications. For instance, an HRD may be used to determine whetherType I bitstreams and Type II bitstreams conform to HEVC specifications.A Type I bitstream may contain only Video Coding Layer (VCL) NetworkAccess Layer (NAL) units and filler data NAL units. A Type II bitstreammay contain additional other NAL units and syntax elements.

Joint Collaborative Team on Video Coding (JCTVC) document JCTVC-I0333includes sub-picture based HRD and supports picture timing SEI messages.This functionality has been incorporated into the High Efficiency VideoCoding (HEVC) Committee Draft (JCTVC-I1003).

One example of modified syntax and semantics in accordance with thesystems and methods disclosed herein is given in Table (1) as follows.Modifications in accordance with the systems and methods disclosedherein are denoted in bold.

TABLE 1 pic_timing( payloadSize ) {  if ( CpbDpbDelaysPresentFlag ) {  cpb_removal_delay   dpb_output_delay   if (sub_pic_cpb_params_present_flag ) {    num_decoding_units_minus1   common du cpb removal delay flag   if(common_du_cpb_removal_delay_flag) {    common_du_cpb_removal_delay   }   for(i = 0; i <= num_decoding_units_minus1; i++){    num_nalus_in_du_minus1[ i ]    if(!common_du_cpb_removal_delay_flag)    du_cpb_removal_delay[ i ]    }   }  } }

Examples regarding buffering period SEI message semantics in accordancewith the systems and methods disclosed herein are given as follows. Inparticular, additional detail regarding the semantics of the modifiedsyntax elements are given as follows. When NalHrdBpPresentFlag orVclHrdBpPresentFlag are equal to 1, a buffering period SEI message canbe associated with any access unit in the bitstream, and a bufferingperiod SEI message may be associated with each Instantaneous DecodingRefresh (IDR) access unit, with each Clean Random Access (CRA) accessunit, and with each access unit associated with a recovery point SEImessage. For some applications, the frequent presence of a bufferingperiod SEI message may be desirable. A buffering period is specified asthe set of access units between two instances of the buffering periodSEI message in decoding order. seq_parameter_set_id specifies thesequence parameter set that contains the sequence HRD attributes. Thevalue of seq_parameter_set_id may be equal to the value ofseq_parameter_set_id in the picture parameter set referenced by theprimary coded picture associated with the buffering period SEI message.The value of seq_parameter_set_id may be in the range of 0 to 31,inclusive.

initial_cpb_removal_delay[SchedSelIdx] specifies the delay for theSchedSelIdx-th CPB between the time of arrival in the CPB of the firstbit of the coded data associated with the access unit associated withthe buffering period SEI message and the time of removal from the CPB ofthe coded data associated with the same access unit, for the firstbuffering period after HRD initialisation. The syntax element has alength in bits given by initial_cpb_removal_delay_length_minus1+1. It isin units of a 90 kHz clock. initial_cpb_removal_delay[SchedSelIdx] maynot be equal to 0 and may not exceed90000*(CpbSize[SchedSelIdx]/BitRate[SchedSelIdx]), the time-equivalentof the CPB size in 90 kHz clock units.

initial_cpb_removal_delay_offset[SchedSelIdx] is used for theSchedSelIdx-th CPB in combination with the cpb_removal_delay to specifythe initial delivery time of coded access units to the CPB.initial_cpb_removal_delay_offset[SchedSelIdx] is in units of a 90 kHzclock. The initial_cpb_removal_delay_offset[SchedSelIdx] syntax elementis a fixed length code whose length in bits is given byinitial_cpb_removal_delay_length_minus1+1. This syntax element is notused by decoders and is needed only for the delivery scheduler (HSS)(e.g., as specified in Annex C of JCTVC-I1003).

Over the entire coded video sequence, the sum ofinitial_cpb_removal_delay[SchedSelIdx] andinitial_cpb_removal_delay_offset[SchedSelIdx] may be constant for eachvalue of SchedSelIdx.

initial_du_cpb_removal_delay[SchedSelIdx] specifies the delay for theSchedSelIdx-th CPB between the time of arrival in the CPB of the firstbit of the coded data associated with the first decoding unit in theaccess unit associated with the buffering period SEI message and thetime of removal from the CPB of the coded data associated with the samedecoding unit, for the first buffering period after HRD initialization.The syntax element has a length in bits given byinitial_cpb_removal_delay_length_minus1+1. It is in units of a 90 kHzclock. initial_du_cpb_removal_delay[SchedSelIdx] may not be equal to 0and may not exceed 90000*(CpbSize[SchedSelIdx]/BitRate[SchedSelIdx]),the time-equivalent of the CPB size in 90 kHz clock units.

initial du_cpb_removal_delay_offset[SchedSelIdx] is used for theSchedSelIdx-th CPB in combination with the cpb_removal_delay to specifythe initial delivery time of decoding units to the CPB.initial_cpb_removal_delay_offset[ SchedSelIdx] is in units of a 90 kHzclock. The initial_du_cpb_removal_delay_offset[SchedSelIdx] syntaxelement is a fixed length code whose length in bits is given byinitial_cpb_removal_delay_length_minus1+1. This syntax element is notused by decoders and is needed only for the delivery scheduler (HSS)(e.g., as specified in Annex C of JCTVC-I1003).

Over the entire coded video sequence, the sum of

initial_du_cpb_removal_delay[SchedSelIdx] and

initial_du_cpb_removal_delay_offset[SchedSelIdx] may be constant foreach value of SchedSelIdx.

Examples regarding picture timing SEI message semantics in accordancewith the systems and methods disclosed herein are given as follows. Inparticular, additional detail regarding the semantics of the modifiedsyntax elements are given as follows.

The syntax of the picture timing SEI message is dependent on the contentof the sequence parameter set that is active for the coded pictureassociated with the picture timing SEI message. However, unless thepicture timing SEI message of an Instantaneous Decoding Refresh (IDR)access unit is preceded by a buffering period SEI message within thesame access unit, the activation of the associated sequence parameterset (and, for IDR pictures that are not the first picture in thebitstream, the determination that the coded picture is an IDR picture)does not occur until the decoding of the first coded slice NetworkAbstraction Layer (NAL) unit of the coded picture. Since the coded sliceNAL unit of the coded picture follows the picture timing SEI message inNAL unit order, there may be cases in which it is necessary for adecoder to store the raw byte sequence payload (RBSP) containing thepicture timing SEI message until determining the parameters of thesequence parameter that will be active for the coded picture, and thenperform the parsing of the picture timing SEI message.

The presence of picture timing SEI message in the bitstream is specifiedas follows. If CpbDpbDelaysPresentFlag is equal to 1, one picture timingSEI message may be present in every access unit of the coded videosequence. Otherwise (CpbDpbDelaysPresentFlag is equal to 0), no picturetiming SEI messages may be present in any access unit of the coded videosequence.

cpb_removal_delay specifies how many clock ticks (see subclause E.2.1 ofJCTVC-11003) to wait after removal from the CPB of the access unitassociated with the most recent buffering period SEI message in apreceding access unit before removing from the buffer the access unitdata associated with the picture timing SEI message. This value is alsoused to calculate an earliest possible time of arrival of access unitdata into the CPB for the HSS, as specified in Annex C of JCTVC-I1003.The syntax element is a fixed length code whose length in bits is givenby cpb_removal_delay_length_minus1+1. The cpb_removal_delay is theremainder of a modulo 2^((cpb) ^(_) ^(removal) ^(_) ^(delay) ^(_)^(length) ^(_) ^(minus1+1)) counter.

The value of cpb_removal_delay_length_minus1 that determines the length(in bits) of the syntax element cpb_removal_delay is the value ofcpb_removal_delay_length_minus1 coded in the sequence parameter set thatis active for the primary coded picture associated with the picturetiming SEI message, although cpb_removal_delay specifies a number ofclock ticks relative to the removal time of the preceding access unitcontaining a buffering period SEI message, which may be an access unitof a different coded video sequence. dpb_output_delay is used to computethe Decoded Picture Buffer (DPB) output time of the picture. Itspecifies how many clock ticks to wait after removal of the lastdecoding unit in an access unit from the CPB before the decoded pictureis output from the DPB (see subclause C.2 of JCTVC-I1003).

With respect to the DPB, a picture is not removed from the DPB at itsoutput time when it is still marked as “used for short-term reference”or “used for long-term reference”. Only one dpb_output_delay isspecified for a decoded picture. The length of the syntax elementdpb_output_delay is given in bits by dpb_output_delay_length_minus1+1.When max_dec_pic_buffering[max_temporal_layers_minus1] is equal to 0,dpb_output_delay may be equal to 0.

The output time derived from the dpb_output_delay of any picture that isoutput from an output timing conforming decoder as specified insubclause C.2 of JCTVC-I1003 may precede the output time derived fromthe dpb_output_delay of all pictures in any subsequent coded videosequence in decoding order. The picture output order established by thevalues of this syntax element may be the same order as established bythe values of PicOrderCnt( ) as specified by subclause. For picturesthat are not output by the “bumping” process of subclause because theyprecede, in decoding order, an IDR picture withno_output_of_prior_pics_flag equal to 1 or inferred to be equal to 1,the output times derived from dpb_output_delay may be increasing withincreasing value of PicOrderCnt( ) relative to all pictures within thesame coded video sequence.

num_decoding_units_minus1 plus 1 specifies the number of decoding unitsin the access unit the picture timing SEI message is associated with.The value of num_decoding_units_minus1 may be in the range of 0 toPicWidthInCtbs*PicHeightInCtbs−1, inclusive.

common_du_cpb_removal_delay_flag equal to 1 specifies that the syntaxelement common_du_cpb_removal_delay is present.common_du_cpb_removal_delay_flag equal to 0 specifies that the syntaxelement common_du_cpb_removal_delay is not present.

common_du_cpb_removal_delay specifies information as follows: If adecoding unit is the first decoding unit in the access unit associatedwith the picture timing SEI message then common_du_cpb_removal_delayspecifies how many sub-picture clock ticks (see subclause E.2.1 ofJCTVC-I1003) to wait after removal from the CPB of the last decodingunit in the access unit associated with the most recent buffering periodSEI message in a preceding access unit before removing from the CPB thefirst decoding unit in the access unit associated with the picturetiming SEI message.

Otherwise, common_du_cpb_removal_delay specifies how many sub-pictureclock ticks (see subclause E.2.1 of JCTVC-I1003) to wait after removalfrom the CPB of the preceding decoding unit in the access unitassociated with the picture timing SEI message before removing from theCPB the current decoding unit in the access unit associated with thepicture timing SEI message. This value is also used to calculate anearliest possible time of arrival of decoding unit data into the CPB forthe HSS, as specified in Annex C. The syntax element is a fixed lengthcode whose length in bits is given by cpb_removal_delay_length_minus1+1.The common_du_cpb_removal_delay is the remainder of a modulo 2^((cpb)^(_) ^(removal) ^(_) ^(delay) ^(_) ^(length) ^(_) ^(minus1+1)) counter.

An alternate way of specifying common_du_cpb_removal_delay is asfollows:

common_du_cpb_removal_delay specifies how many sub-picture clock ticks(see subclause E.2.1 of JCTVC-I1003) to wait after removal from the CPBof the last decoding unit before removing from the CPB the currentdecoding unit in the access unit associated with the picture timing SEImessage. This value is also used to calculate an earliest possible timeof arrival of decoding unit data into the CPB for the HSS, as specifiedin Annex C. The syntax element is a fixed length code whose length inbits is given by cpb_removal_delay_length_minus1+1. Thecommon_du_cpb_removal_delay is the remainder of a modulo 2^((cpb) ^(_)^(removal) ^(_) ^(delay) ^(_) ^(length) ^(_) ^(minus1+1)) counter.

The value of cpb_removal_delay_length_minus1 that determines the length(in bits) of the syntax element common_du_cpb_removal_delay is the valueof cpb_removal_delay_length_minus1 coded in the sequence parameter setthat is active for the coded picture associated with the picture timingSEI message, although common_du_cpb_removal_delay specifies a number ofsub-picture clock ticks relative to the removal time of the firstdecoding unit in the preceding access unit containing a buffering periodSEI message, which may be an access unit of a different coded videosequence.

num_nalus_in_du_minus1[i] plus 1 specifies the number of NAL units inthe i-th decoding unit of the access unit the picture timing SEI messageis associated with. The value of num_nalus_in_du_minus1[i] may be in therange of 0 to PicWidthInCtbs*PicHeightInCtbs−1, inclusive.

The first decoding unit of the access unit consists of the firstnum_nalus_in_du_minus1[0]+1 consecutive NAL units in decoding order inthe access unit. The i-th (with i greater than 0) decoding unit of theaccess unit consists of the num_nalus_in_du_minus1[i]+1 consecutive NALunits immediately following the last NAL unit in the previous decodingunit of the access unit, in decoding order. There may be at least oneVCL NAL unit in each decoding unit. All non-VCL NAL units associatedwith a VCL NAL unit may be included in the same decoding unit.

du_cpb_removal_delay[i] specifies how many sub-picture clock ticks (seesubclause E.2.1 of JCTVC-I1003) to wait after removal from the CPB ofthe first decoding unit in the access unit associated with the mostrecent buffering period SEI message in a preceding access unit beforeremoving from the CPB the i-th decoding unit in the access unitassociated with the picture timing SEI message. This value is also usedto calculate an earliest possible time of arrival of decoding unit datainto the CPB for the HSS (e.g., as specified in Annex C of JCTVC-I1003).The syntax element is a fixed length code whose length in bits is givenby cpb_removal_delay_length_minus1+1. The du_cpb_removal_delay[i] is theremainder of a modulo 2^((cpb) ^(_) ^(removal) ^(_) ^(delay) ^(_)^(length) ^(_) ^(minus1+1)) counter.

The value of cpb_removal_delay_length_minus1 that determines the length(in bits) of the syntax element du_cpb_removal_delay[i] is the value ofcpb_removal_delay_length_minus1 coded in the sequence parameter set thatis active for the coded picture associated with the picture timing SEImessage, although du_cpb_removal_delay[i] specifies a number ofsub-picture clock ticks relative to the removal time of the firstdecoding unit in the preceding access unit containing a buffering periodSEI message, which may be an access unit of a different coded videosequence.

In one configuration, the timing of decoding unit removal and decodingof decoding units may be implemented as follows.

If SubPicCpbFlag is equal to 0, the variable CpbRemovalDelay(m) is setto the value of cpb_removal_delay in the picture timing SEI messageassociated with the access unit that is decoding unit m, and thevariable T_(c) is set to t_(c). Otherwise if SubPicCpbFlag is equal to 1and common_du_cpb_removal_delay_flag is 0 the variableCpbRemovalDelay(m) is set to the value of du_cpb_removal_delay[i] fordecoding unit m (with m ranging from 0 to num_decoding_unitsminus1) inthe picture timing SEI message associated with the access unit thatcontains decoding unit m, and the variable T_(c) is set to t_(c) _(_)_(sub).

In some cases, Otherwise if SubPicCpbFlag is equal to 1 andcommon_du_cpb_removal_delay_flag is 0, the variable CpbRemovalDelay(m)is set to the value of (m+1)*du_cpb_removal_delay[i] for decoding unit m(with m ranging from 0 to num_decoding_units_minus1) in the picturetiming SEI message associated with the access unit that containsdecoding unit m, and the variable T_(c) is set to t_(c) _(_) _(sub).

Otherwise, if SubPicCpbFlag is equal to 1 andcommon_du_cpb_removal_delay_flag is 1, the variable CpbRemovalDelay(m)is set to the value of common_du_cpb_removal_delay for decoding unit min the picture timing SEI message associated with the access unit thatcontains decoding unit m, and the variable T_(c) is set to t_(c) _(_)_(sub).

When a decoding unit m is the decoding unit with n equal to 0 (the firstdecoding unit of the access unit that initializes the HRD), the nominalremoval time of the decoding unit from the CPB is specified byt_(r,n)(0)=InitCpbRemovalDelay[ SchedSelIdx]/90000.

When a decoding unit m is the first decoding unit of the first accessunit of a buffering period that does not initialize the HRD, the nominalremoval time of the decoding unit from the CPB is specified byt_(r,n)(m)=t_(r,n)(m_(b))+T_(c)*CpbRemovalDelay(m), where t_(r,n)(m_(b))is the nominal removal time of the first decoding unit of the previousbuffering period.

When a decoding unit m is the first decoding unit of a buffering period,m_(b) is set equal to m at the removal time t_(r,n)(m) of the decodingunit m. The nominal removal time t_(r,n)(m) of a decoding unit m that isnot the first decoding unit of a buffering period is given byt_(r,n)(m)=t_(r,n)(m_(b))+T_(c)*CpbRemovalDelay(m), where t_(r,n)(m_(b))is the nominal removal time of the first decoding unit of the currentbuffering period.

The removal time of decoding unit m is specified as follows. Iflow_delay_hrd_flag is equal to 0 or t_(r,n)(m)>=t_(af)(m), the removaltime of decoding unit m is specified by t_(r)(m)=t_(r,n)(m). Otherwise(low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m)), the removaltime of decoding unit m is specified byt_(r)(m)=t_(r,n)(m)+T_(c)*Ceil((t_(af)(m)−t_(r,n)(m))/T_(c)). The lattercase (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))indicates that the size of decoding unit m, b(m), is so large that itprevents removal at the nominal removal time.

In another case the removal time of decoding unit m is specified asfollows. If low_delay_hrd_flag is equal to 0 or t_(r,n)(m)>=t_(af)(m),the removal time of decoding unit m is specified by t_(r)(m)=t_(r,n)(m).Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m)),the removal time of decoding unit m which is not the last decoding unitin the access unit is specified by t_(r)(m)=t_(af)(m), and the removaltime of decoding unit m which is the last decoding unit in the accessunit t_(r)(m)=t_(r,n)(m)+T_(c)*Ceil((t_(af)(m)−t_(r,n)(m))/T_(c)). Thelatter case (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))indicates that the size of decoding unit m, b(m), is so large that itprevents removal at the nominal removal time.

In another case the removal time of decoding unit m is specified asfollows. If low_delay_hrd_flag is equal to 0 or t_(r,n)(m)>=t_(af)(m),the removal time of decoding unit m is specified byt_(r)(m)=t_(r,n)(m)). Otherwise (low_delay_hrd_flag is equal to 1 andt_(r,n)(m)<t_(af)(m)), the removal time of decoding unit m which is notthe last decoding unit in the access unit is specified byt_(r)(m)=t_(af)(m), and the removal time of decoding unit m which is thelast decoding unit in the access unitt_(r)(m)=t_(r,n)(m)+t_(c)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c)). The lattercase (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))indicates that the size of decoding unit m, b(m), is so large that itprevents removal at the nominal removal time.

In another case the removal time of decoding unit m is specified asfollows. If low_delay_hrd_flag is equal to 0 or t_(r,n)(m)>=t_(af)(m),the removal time of decoding unit m is specified by t_(r)(m)=t_(r,n)(m).Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m)),the removal time of decoding unit m is specified by t_(r)(m)=t_(af)(m).The latter case (low_delay_hrd_flag is equal to 1 andt_(r,n)(m)<t_(af)(m)) indicates that the size of decoding unit m, b(m),is so large that it prevents removal at the nominal removal time.

When SubPicCpbFlag is equal to 1, the nominal CPB removal time of accessunit n t_(r,n)(n) is set to the nominal CPB removal time of the lastdecoding unit in access unit n, the CPB removal time of access unit nt_(r)(n) is set to the CPB removal time of the last decoding unit inaccess unit n.

When SubPicCpbFlag is equal to 0, each decoding unit is an access unit,hence the nominal CPB removal time and the CPB removal time of accessunit n are the nominal CPB removal time and the CPB removal time ofdecoding unit n.

At CPB removal time of decoding unit m, the decoding unit isinstantaneously decoded.

Another example of modified syntax and semantics for a picture timingSEI message in accordance with the systems and methods disclosed hereinis given in Table (2) as follows.

Modifications in accordance with the systems and methods disclosedherein are denoted in bold.

TABLE 2 pic_timing( payloadSize ) {  if( CpbDpbDelaysPresentFlag ) {  cpb_removal_delay   dpb_output_delay   if(sub_pic_cpb_params_present_flag ) {    num_decoding_units_minus1   common_du_cpb_removal_delay_flag   if(common_du_cpb_removal_delay_flag) {     common num nalus in duminus1     common du cpb removal delay    }   for ( i = 0; i <=num_decoding_units_minus1; i++) {    num_nalus_in_du_minus1[ i ]   if(!common_du_cpb_removal_delay_flag)     du_cpb_removal_delay[ i ]   }   }  } }

The illustrated example in Table (2) includes a syntax elementcommon_num_nalus_in_du_minus1, which may be used to determine how muchdata should be removed from the CPB when removing a decoding unit.common_num_nalus_in_du_minus1 plus 1 specifies the number of NAL unitsin each decoding unit of the access unit the picture timing SEI messageis associated with. The value of common_num_nalus_in_du_minus1 may be inthe range of 0 to PicWidthInCtbs*PicHeightInCtbs−1, inclusive.

The first decoding unit of the access unit consists of the firstcommon_num_nalus_in_du_minus1+1 consecutive NAL units in decoding orderin the access unit. The i-th (with i greater than 0) decoding unit ofthe access unit consists of the commonnum_nalus_in_du_minus1+1consecutive NAL units immediately following the last NAL unit in theprevious decoding unit of the access unit, in decoding order. There maybe at least one VCL NAL unit in each decoding unit. All non-VCL NALunits associated with a VCL NAL unit may be included in the samedecoding unit.

Another example of modified syntax and semantics for a picture timingSEI message in accordance with the systems and methods disclosed hereinis given in Table (3) as follows. Modifications in accordance with thesystems and methods disclosed herein are denoted in bold.

TABLE 3 pic_timing( payloadSize ) {  if( CpbDpbDelaysPresentFlag ) {  cpb_removal_delay   dpb_output_delay   if(sub_pic_cpb_params_present_flag ) {    num_decoding_units_minus1    common num nalus in du flag    if(common nalus in du flag) {    common_num_nalus_in_du minus1    }     common_ducpb_removal_delay_flag    if(common_du_cpb_removal_delay_flag) {    common_du_cpb_removal_delay    }    for ( i = 0; i <=num_decoding_units_minus1; i++) {      if(!common_num_nalus_in_du_flag)    num_nalus_in_du_minus1[ i ]     if(!common_du_cpb_removal_delay_flag)     du_cpb_removal_delay[ i ]   }   }  } }

The illustrated example in Table (3) includes a syntax elementcommon_num_nalus_in_du_flag that, when equal to 1, specifies that thesyntax element common_num_nalus_in_du_minus1 is present.common_num_nalus_in_du_flag equal to 0 specifies that the syntax elementcommon_num_nalus_in_du_minus1 is not present.

In yet another embodiment flags common_du_cpb_removal_delay_flagcommon_num_nalusin_du_minus1, may not be sent. Instead syntax elementscommon_num_nalus_in_du_minus1 and common_du_cpb_removal_delay could besent every time. In this case, a value of 0 (or some other) for thesesyntax elements could be used to indicate that these elements are notsignaled.

In addition to modifications to the syntax elements and semantics of thepicture timing SEI message, the present systems and methods may alsoimplement a bitstream constraint so that sub-picture based CPB operationand access unit level CPB operation result in the same timing ofdecoding unit removal.

When sub_pic_cpb_params_present_flag equals to 1 that sub-picture levelCPB removal delay parameters are present, the CPB may operate at accessunit level or sub-picture level. sub_pic_cpb_params_present_flag equalto 0 specifies that sub-picture level CPB removal delay parameters arenot present and the CPB operates at access unit level. Whensub_pic_cpb_params_present_flag is not present, its value is inferred tobe equal to 0.

To support the operation at both access unit level or sub-picture level,the following bitstream constraints may be used: Ifsub_pic_cpb_params_present_flag is 1, then it is requirement ofbitstream conformance that the following constraint is obeyed whensignaling the values for cpb_removal_delay and du_cpb_removal_delay[i]for all i:

$\begin{matrix}{{{cpb\_ removal}{\_ delay}} = \frac{\begin{matrix}{\left( {\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_ minus}\; 1}{{du\_ cpb}{\_ removal}{{\_ delay}\lbrack i\rbrack}}} \right)*} \\t_{c,{sub}}\end{matrix}}{t_{c}}} & \left\lbrack {{Math}.\mspace{14mu} 3} \right\rbrack\end{matrix}$where du_cpb_removal_delay[i] are the decoding unit CPB removal delayparameters, t_(c) is a clock tick, t_(c) _(_) _(sub) is sub-pictureclock tick, num_decoding_units_minus1 is an amount of decoding units inthe access unit offset by one, and i is an index. In some embodiments atolerance parameter could be added to satisfy the above constraint.

To support the operation at both access unit level or sub-picture level,the bitstream constraints as follows may be used: Let the variableT_(du)(k) be defined as:

$\begin{matrix}{{T_{du}(k)} = {{T_{du}\left( {k - 1} \right)} + {t_{c\_ sub}*{\quad{\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_ minus}\; 1_{k}}\left( {{{du\_ cpb}{\_ removal}{\_ delay}{\_ minus}\;{1_{k}\lbrack i\rbrack}} + {\quad\left. \quad 1 \right)}} \right.}}}}} & \left\lbrack {{Math}.\mspace{14mu} 4} \right\rbrack\end{matrix}$where du_cpb_removal_delay_minus1_(k)[i] andnum_decoding_units_minus1_(k) are parameters for i′th decoding unit ofk′th access unit (with k=0 for the access unit that initialized the HRDand T_(du)(k)=0 for k<1), and wheredu_cpb_removal_delay_minus1_(k)[i]+1=du_cpb_removal_delay_(k)[i] is thedecoding unit CPB removal delay parameter for the l′th decoding unit ofthe k′th access unit, and num_decoding_units_minus1_(k) is the number ofdecoding units in the k′th access unit, t_(c) is a clock tick, t_(c,sub)is a sub-picture clock tick, and i and k are an indices. Then when thepicture timing flag (e.g., sub_pic_cpb_params_present_flag) is set to 1,the following constraint shall be true:(au_cpb_removal_delay_minus1+1)*t_(c)=T_(du)(k), where(au_cpb_removal_delay_minus1+1)=cpb_removal_delay, the CPB removaldelay. Thus in this case the CPB removal delay(au_cpb_removal_delay_minus1+1) is set such that the operation ofsub-picture based CPB operation and access unit based CPB operationresult in the same timing of access unit removal and last decoding unitof the access unit removal.

To support the operation at both access unit level or sub-picture level,the following bitstream constraints may be used: Ifsubpic_cpb_params_present_flag is 1, then it is a requirement ofbitstream conformance that the following constraint is obeyed whensignaling the values for cpb_removal_delay and du_cpb_removal_delay[i]for all i:

$\begin{matrix}{{- 1} \leq \left\lbrack {{{cpb\_ removal}{\_ delay}*t_{c}} - \left( {\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_ minus}\; 1}{{du\_ cpb}{\_ removal}{{\_ delay}\lbrack i\rbrack}*t_{c,{sub}}}} \right)} \right\rbrack \leq 1} & \left\lbrack {{Math}.\mspace{14mu} 5} \right\rbrack\end{matrix}$where du_cpb_removal_delay[i] are the decoding unit CPB removal delayparameters, t_(c) is a clock tick, t_(c,sub) is sub-picture clock tick,num_decoding_units_minus1 is an amount of decoding units in the accessunit offset by one and i is an index.

To support the operation at both access unit level or sub-picture level,the following bitstream constraints may be used: Ifsub_pic_cpb_params_present_flag is 1, then it is requirement ofbitstream conformance that the following constraint is obeyed whensignaling the values for cpb_removal_delay and

du_cpb_removal_delay[num_decoding_units_minus1]:cpb_removal_delay*t_(c)=du_cpb_removal_delay[num_decoding_units_minus1]*t_(c,sub)where

du_cpb_removal_delay[num_decoding_units_minus1] is the decoding unit CPBremoval delay parameter for the num_decoding_units_minus1′th decodingunit, t_(c) is a clock tick, t_(c,sub) is a sub-picture clock ticknum_decoding_units_minus1 is an amount of decoding units in the accessunit offset by one. In some embodiments a tolerance parameter could beadded to satisfy the above constraint.

To support the operation at both access unit level or sub-picture level,the following bitstream constraints may be used: Ifsub_pic_cpb_params_present_flag is 1, then it is requirement ofbitstream conformance that the following constraint is obeyed whensignaling the values for cpb_removal_delay and du_cpb_removal_delay[i]for all i:

-   −1<=(cpb_removal_delay*t_(c)−du_cpb_removal_delay[num_decoding_units_minus1]*t_(c,sub))<=1,where    du_cpb_removal_delay[num_decoding_units_minus1] is the decoding unit    CPB removal delay parameter for the num_decoding_units_minus1′th    decoding unit, t_(c) is a clock tick, t_(c,sub) is a sub-picture    clock tick num_decoding_units_minus1 is an amount of decoding units    in the access unit offset by one.

Additionally, the present systems and methods may modify the timing ofdecoding unit removal. When sub-picture level CPB removal delayparameters are present, the removal time of decoding unit for “bigpictures” (when low_delay_hrd_flag is 1 and t_(r,n)(m)<t_(af)(m)) may bechanged to compensate for difference that can arise due to clock tickcounter and sub-picture clock tick counter.

When sub_pic_cpb_params_present_flag equals to 1 then sub-picture levelCPB removal delay parameters are present and the CPB may operate ataccess unit level or sub-picture level. sub_pic_cpb_params_present_flagequal to 0 specifies that sub-picture level CPB removal delay parametersare not present and the CPB operates at access unit level. Whensub_pic_cpb_params_present_flag is not present, its value is inferred tobe equal to 0.

Specifically, one example of timing of decoding unit removal anddecoding of decoding unit implementation is as follows. The variableSubPicCpbPreferredFlag is either specified by external means, or whennot specified by external means, set to 0. The variable SubPicCpbFlag isderived as follows: SubPicCpbFlag=SubPicCpbPreferredFlag &&sub_pic_cpb_params_present_flag. If SubPicCpbFlag is equal to 0, the CPBoperates at access unit level and each decoding unit is an access unit.Otherwise the CPB operates at sub-picture level and each decoding unitis a subset of an access unit.

If SubPicCpbFlag is equal to 0, the variable CpbRemovalDelay(m) is setto the value of cpb_removal_delay in the picture timing SEI messageassociated with the access unit that is decoding unit m, and thevariable T_(c) is set to t_(c). Otherwise the variableCpbRemovalDelay(m) is set to the value of du_cpb_removal_delay[i] fordecoding unit m in the picture timing SEI message associated with theaccess unit that contains decoding unit m, and the variable T_(c) is setto t_(c) _(_) _(sub).

When a decoding unit m is the decoding unit with n equal to 0 (the firstdecoding unit of the access unit that initializes the HRD), the nominalremoval time of the decoding unit from the CPB is specified byt_(r,n)(0)=InitCpbRemovalDelay[SchedSelIdx]/90000.

When a decoding unit m is the first decoding unit of the first accessunit of a buffering period that does not initialize the HRD, the nominalremoval time of the decoding unit from the CPB is specified byt_(r,n)(m)=t_(r,n)(m_(b))+T_(c)*CpbRemovalDelay(m) where t_(r,n)(m_(b))is the nominal removal time of the first decoding unit of the previousbuffering period.

When a decoding unit m is the first decoding unit of a buffering period,m_(b) is set equal to m at the removal time t_(r,n)(m) of the decodingunit m.

The nominal removal time t_(r,n)(m) of a decoding unit m that is not thefirst decoding unit of a buffering period is given byt_(r,n)(m)=t_(r,n)(m_(b))+T_(c)*CpbRemovalDelay(m) where t_(r,n)(m_(b))is the nominal removal time of the first decoding unit of the currentbuffering period.

The removal time of decoding unit m is specified as follows. Thevariable ClockDiff is defined as ClockDiff=

(num_units_in_tick−(num_units_in_sub_tick*(num_decoding_units_minus1+1))/time_scale).In some case it may be requirement of a bitstream conformance that theparameters num_units_in_tick, num_units_in_sub_ticknum_decoding_units_minus1 are signaled such that following equation issatisfied.(num_units_intick−(num_units_in_sub_tick*(num_decoding_units_minus1+1)))>=0

In some other case it may be requirement of a bitstream conformance thatthe parameters num_units_in_tick, num_units_insub_tick,num_decoding_units_minus1 may be signaled such that following equationis satisfied.

(num_units_in_tick−(num_units_in_sub_tick*(num_decoding_units_minus1+1)))<=0If low_delay_hrd_flag is equal to 0 or t_(r,n)(m)>=t_(af)(m), theremoval time of decoding unit m is specified by t_(r)(m)=t_(r,n)(m).

Otherwise, (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m)),and when sub_pic_cpb_params_present_flag equals to 1 and the CPB isoperating at sub-picture level, and if ClockDiff is greater than zero,the removal time of decoding unit m when it is the last decoding unit ofthe access unit n is specified byt_(r)(m)=t_(r,n)(m)+T_(c)*Ceil((t_(af)(m)−t_(r,n)(m))T_(c))+ClockDiff.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m)),and when sub_pic_cpb_params_present_flag equals to 1 and the CPB isoperating at access unit level and if ClockDiff is less than zero theremoval time of access unit n is specified byt_(r)(m)=t_(r,n)(m)+t_(c)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c))−ClockDiff.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m))<t_(af)(m)),the removal time of decoding unit m is specified byt_(r)(m)=t_(r,n)(m)+T_(c)*Ceil((t_(af)(m)−t_(r,n)(m))/T_(c)). The lattercase (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))indicates that the size of decoding unit m, b(m), is so large that itprevents removal at the nominal removal time. Otherwise(low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m)) and when apicture timing flag is set to 1 and the CPB is operating at sub-picturelevel, the removal time for the last decoding unit m of access unit,t_(r)(m) according to: t_(r)(m)=t_(r,n)(m)+min((t_(c) _(_)_(sub)*Ceil(t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c)))) where t_(r,n)(m) is thenominal removal time of the last decoding unit m, t_(c,sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(n)<t_(af)(n))and when a picture timing flag is set to 1 and the CPB is operating ataccess unit level, the removal time for access unit n, t_(r)(n)according to: t_(r)(n)=t_(r,n)(n)+min((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))) where t_(r,n)(m) is thenominal removal time of the last decoding unit n, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))and a picture timing flag is set to 1 and the CPB is operating atsub-picture level, the removal time for the last decoding unit m ofaccess unit, t_(r)(m) according to:t_(r)(m)=t_(r,n)(m)+(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit in,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick and t_(af)(n) is a final arrival time of access unit n.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(n)<t_(af)(n))and a picture timing flag is set to 1 and the CPB is operating at accessunit level, the removal time for access unit n, t_(r)(n) according to:t_(r)(n)=t_(r,n)(n)+*Ceil((t_(af)(n)−n)) t_(r,n)(n))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit n,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit in,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick and t_(af)(n) is a final arrival time of access unit n.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))and a picture timing flag is set to 1 and the CPB is operating atsub-picture level, the removal time for the decoding unit which is notthe last decoding unit of the access unit is set as t_(r)(m)=t_(af)(m),where t_(af)(m) is a final arrival time of decoding unit m. And theremoval time of the last decoding unit m of access unit, t_(r)(m) is setaccording to: t_(r)(m)=t_(r,n)(m)+(t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))) where t_(r,n)(m)is the nominal removal time of the last decoding unit m, t_(c) _(_)_(sub) is sub-picture clock tick, Ceil( ) is a ceiling function,t_(af)(m) is a final arrival time of last decoding unit m, t_(r,n)(n) isthe nominal removal time of the access unit n, t_(c) is clock tick andt_(af)(n) is a final arrival time of access unit n and t_(af)(m) is afinal arrival time of the last decoding unit m in the access unit n.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))and a picture timing flag is set to 1 and the CPB is operating atsub-picture level, the removal time for the decoding unit which is notthe last decoding unit of the access unit is set as t_(r)(m)=t_(af)(m),where t_(af)(m) is a final arrival time of decoding unit m. And theremoval time of the last decoding unit m of access unit, t_(r)(m) is setaccording to:t_(r)(m)=t_(r,n)(m)+(t_(c)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit m,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick and t_(af)(n) is a final arrival time of access unit n andt_(af)(m) is a final arrival time of the last decoding unit m in theaccess unit n.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(m)<t_(af)(m))and a picture timing flag is set to 1 and the CPB is operating atsub-picture level, the removal time for the decoding unit is set ast_(r)(m)=t_(af)(m) where t_(r,n)(m) is the nominal removal time of thedecoding unit m, t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) isa ceiling function, t_(af)(m) is a final arrival time of decoding unitm, t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick and t_(af)(n) is a final arrival time of access unit n andt_(af)(m) is a final arrival time of the decoding unit m in the accessunit n.

Otherwise (low_delay_hrd_flag is equal to 1 and t_(r,n)(n)<t_(af)(n))and a picture timing flag is set to 1 and the CPB is operating at accessunit level, the removal time for access unit n, t_(r)(n) according to:t_(r)(n)=t_(af)(n) where t_(r,n)(m) is the nominal removal time of thelast decoding unit n, t_(c) _(_) _(sub) is sub-picture clock tick, Ceil() is a ceiling function, t_(af)(m) is a final arrival time of lastdecoding unit m, t_(r,n)(n) is the nominal removal time of the accessunit n, t_(c) is clock tick and t_(af)(n) is a final arrival time ofaccess unit n.

When SubPicCpbFlag is equal to 1, the nominal CPB removal time of accessunit n t_(r,n)(n) is set to the nominal CPB removal time of the lastdecoding unit in access unit n, the CPB removal time of access unit nt_(r)(n) is set to the CPB removal time of the last decoding unit inaccess unit n.

When SubPicCpbFlag is equal to 0, each decoding unit is an access unit,hence the nominal CPB removal time and the CPB removal time of accessunit n are the nominal CPB removal time and the CPB removal time ofdecoding unit n. At CPB removal time of decoding unit m, the decodingunit is instantaneously decoded.

As illustrated by the foregoing, the systems and methods disclosedherein provide syntax and semantics that modify a picture timing SEImessage bitstreams carrying sub-picture based parameters. In someconfigurations, the systems and methods disclosed herein may be appliedto HEVC specifications.

For convenience, several definitions are given as follows, which may beapplied to the systems and methods disclosed herein. A random accesspoint may be any point in a stream of data (e.g., bitstream) wheredecoding of the bitstream does not require access to any point in abitstream preceding the random access point to decode a current pictureand all pictures subsequent to said current picture in output order.

A buffering period may be specified as a set of access units between twoinstances of the buffering period SEI message in decoding order.Supplemental Enhancement Information (SEI) may contain information thatis not necessary to decode the samples of coded pictures from VCL NALunits. SEI messages may assist in procedures related to decoding,display or other purposes. Conforming decoders may not be required toprocess this information for output order conformance to HEVCspecifications (Annex C of HEVC specifications (JCTVC-I1003) includesspecifications for conformance, for example). Some SEI messageinformation may be used to check bitstream conformance and for outputtiming decoder conformance.

A buffering period SEI message may be an SEI message related tobuffering period. A picture timing SEI message may be an SEI messagerelated to CPB removal timing. These messages may define syntax andsemantics which define bitstream arrival timing and coded pictureremoval timing.

A Coded Picture Buffer (CPB) may be a first-in first-out buffercontaining access units in decoding order specified in a hypotheticalreference decoder (HRD). An access unit may be a set of Network AccessLayer (NAL) units that are consecutive in decoding order and containexactly one coded picture. In addition to the coded slice NAL units ofthe coded picture, the access unit may also contain other NAL units notcontaining slices of the coded picture. The decoding of an access unitalways results in a decoded picture. A NAL unit may be a syntaxstructure containing an indication of the type of data to follow andbytes containing that data in the form of a raw byte sequence payloadinterspersed as necessary with emulation prevention bytes.

As used herein, the term “common” generally refers to a syntax elementor a variable that is applicable to more than one thing. For example, inthe context of syntax elements in a picture timing SEI message, the term“common” may mean that the syntax element (e.g.,common_du_cpb_removal_delay) is applicable to all decoding units in anaccess unit associated with the picture timing SEI message.Additionally, units of data are described in terms of “n” and “m”generally refer to access units and decoding units, respectively.

Various configurations are now described with reference to the Figures,where like reference numbers may indicate functionally similar elements.The systems and methods as generally described and illustrated in theFigures herein could be arranged and designed in a wide variety ofdifferent configurations. Thus, the following more detailed descriptionof several configurations, as represented in the Figures, is notintended to limit scope, as claimed, but is merely representative of thesystems and methods.

FIG. 1 is a block diagram illustrating an example of one or moreelectronic devices 102 in which systems and methods for sending amessage and buffering a bitstream may be implemented. In this example,electronic device A 102 a and electronic device B 102 b are illustrated.However, it should be noted that one or more of the features andfunctionality described in relation to electronic device A 102 a andelectronic device B 102 b may be combined into a single electronicdevice in some configurations.

Electronic device A 102 a includes an encoder 104. The encoder 104includes a message generation module 108. Each of the elements includedwithin electronic device A 102 a (e.g., the encoder 104 and the messagegeneration module 108) may be implemented in hardware, software or acombination of both.

Electronic device A 102 a may obtain one or more input pictures 106. Insome configurations, the input picture(s) 106 may be captured onelectronic device A 102 a using an image sensor, may be retrieved frommemory and/or may be received from another electronic device.

The encoder 104 may encode the input picture(s) 106 to produce encodeddata. For example, the encoder 104 may encode a series of input pictures106 (e.g., video). In one configuration, the encoder 104 may be a HighEfficiency Video Coding (HEVC) encoder. The encoded data may be digitaldata (e.g., part of a bitstream 114). The encoder 104 may generateoverhead signaling based on the input signal.

The message generation module 108 may generate one or more messages. Forexample, the message generation module 108 may generate one or more SEImessages or other messages. For a CPB that supports operation on asub-picture level, the electronic device 102 may send sub-pictureparameters, (e.g., CPB removal delay parameter). Specifically, theelectronic device 102 (e.g., the encoder 104) may determine whether toinclude a common decoding unit CPB removal delay parameter in a picturetiming SEI message. For example, the electronic device may set a flag(e.g., common_du_cpb_removal_delay_flag) to one when the encoder 104 isincluding a common decoding unit CPB removal delay parameter (e.g.,common_du_cpb_removal_delay) in the picture timing SEI message. When thecommon decoding unit CPB removal delay parameter is included, theelectronic device may generate the common decoding unit CPB removaldelay parameter that is applicable to all decoding units in an accessunit. In other words, rather than including a decoding unit CPB removaldelay parameter for each decoding unit in an access unit, a commonparameter may apply to all decoding units in the access unit with whichthe picture timing SEI message is associated.

In contrast, when the common decoding unit CPB removal delay parameteris not to be included in the picture timing SEI message, the electronicdevice 102 may generate a separate decoding unit CPB removal delay foreach decoding unit in the access unit with which the picture timing SEImessage is associated. A message generation module 108 may perform oneor more of the procedures described in connection with FIG. 2 and FIG. 3below.

In some configurations, electronic device A 102 a may send the messageto electronic device B 102 b as part of the bitstream 114. In someconfigurations electronic device A 102 a may send the message toelectronic device B 102 b by a separate transmission 110. For example,the separate transmission may not be part of the bitstream 114. Forinstance, a picture timing SEI message or other message may be sentusing some out-of-band mechanism. It should be noted that, in someconfigurations, the other message may include one or more of thefeatures of a picture timing SEI message described above. Furthermore,the other message, in one or more aspects, may be utilized similarly tothe SEI message described above.

The encoder 104 (and message generation module 108, for example) mayproduce a bitstream 114. The bitstream 114 may include encoded picturedata based on the input picture(s) 106. In some configurations, thebitstream 114 may also include overhead data, such as a picture timingSEI message or other message, slice header(s), Picture Parameter Set(s)(PPS(s)), etc. As additional input pictures 106 are encoded, thebitstream 114 may include one or more encoded pictures. For instance,the bitstream 114 may include one or more encoded pictures withcorresponding overhead data (e.g., a picture timing SEI message or othermessage).

The bitstream 114 may be provided to a decoder 112. In one example, thebitstream 114 may be transmitted to electronic device B 102 b using awired or wireless link. In some cases, this may be done over a network,such as the Internet or a Local Area Network (LAN). As illustrated inFIG. 1, the decoder 112 may be implemented on electronic device B 102 bseparately from the encoder 104 on electronic device A 102 a. However,it should be noted that the encoder 104 and decoder 112 may beimplemented on the same electronic device in some configurations. In animplementation where the encoder 104 and decoder 112 are implemented onthe same electronic device, for instance, the bitstream 114 may beprovided over a bus to the decoder 112 or stored in memory for retrievalby the decoder 112.

The decoder 112 may be implemented in hardware, software or acombination of both. In one configuration, the decoder 112 may be a HEVCdecoder. The decoder 112 may receive (e.g., obtain) the bitstream 114.The decoder 112 may generate one or more decoded pictures 118 based onthe bitstream 114. The decoded picture(s) 118 may be displayed, playedback, stored in memory and/or transmitted to another device, etc.

The decoder 112 may include a CPB 120. The CPB 120 may temporarily storeencoded pictures. The CPB 120 may use parameters found in a picturetiming SEI message to determine when to remove data. When the CPB 120supports operation on a sub-picture level, individual decoding units maybe removed rather than entire access units at one time.

The decoder 112 may receive a message (e.g., picture timing SEI messageor other message). The decoder 112 may also determine whether thereceived message includes a common decoding unit CPB removal delayparameter (e.g., common_du_cpb_removal_delay). This may includeidentifying a flag (e.g., common_du_cpb_removal delay_fiag) that is setwhen the common parameter is present in the picture timing SEI message.If the common parameter is present, the decoder 112 may determine thecommon decoding unit CPB removal delay parameter applicable to alldecoding units in the access unit. If the common parameter is notpresent, the decoder 112 may determine a separate decoding unit CPBremoval delay parameter for each decoding unit in the access unit. Thedecoder 112 may also remove decoding units from the CPB 120 using eitherthe common decoding unit CPB removal delay parameter or the separatedecoding unit CPB removal delay parameters. The CPB 120 may perform oneor more of the procedures described in connection with FIG. 4 and FIG. 5below.

The HRD described above may be one example of the decoder 112illustrated in FIG. 1. Thus, an electronic device 102 may operate inaccordance with the HRD and CPB 120 described above, in someconfigurations.

It should be noted that one or more of the elements or parts thereofincluded in the electronic device(s) 102 may be implemented in hardware.For example, one or more of these elements or parts thereof may beimplemented as a chip, circuitry or hardware components, etc. It shouldalso be noted that one or more of the functions or methods describedherein may be implemented in and/or performed using hardware. Forexample, one or more of the methods described herein may be implementedin and/or realized using a chipset, an Application-Specific IntegratedCircuit (ASIC), a Large-Scale Integrated circuit (LSI) or integratedcircuit, etc.

FIG. 2 is a flow diagram illustrating one configuration of a method 200for sending a message. The method 200 may be performed by an encoder 104or one of its sub-parts (e.g., a message generation module 108). Theencoder 104 may determine 202 a picture timing flag (e.g.,sub_pic_cpb_params_present_flag) that indicates whether a CPB 120supports operation on a sub-picture level. For example, when the picturetiming flag is set to 1, the CPB 120 may operate on an access unit levelor a sub-picture level. It should be noted that even when the picturetiming flag is set to 1, the decision about whether to actually operateat the sub-picture level is left to the decoder 112 itself.

The encoder 104 may also determine 204 one or more removal delays fordecoding units in an access unit. For example, the encoder 104 maydetermine a single common decoding unit CPB removal delay parameter(e.g., common_du_cpb_removal_delay) that is applicable to all decodingunits in the access unit from the CPB 120. Alternatively, the encoder104 may determine a separate decoding unit CPB removal delay (e.g.,du_cpb_removal_delay[i]) for each decoding unit in the access unit.

The encoder 104 may also determine 206 one or more NAL parameters thatindicate an amount, offset by one, of NAL units in each decoding unit inthe access point. For example, the encoder 104 may determine a singlecommon NAL parameter (e.g., common_num_nalus_in_du_minus1) that isapplicable to all decoding units in the access unit from the CPB 120.Alternatively, the encoder 104 may determine a separate decoding unitCPB removal delay (e.g., num_nalus_in_du_minus1[i]) for each decodingunit in the access unit.

The encoder 104 may also send 208 a picture timing SEI message thatincludes the picture timing flag, the removal delays and the NALparameters. The picture timing SEI message may also include otherparameters (e.g., cpb_removal_delay, dpb_output_delay, etc). Forexample, the electronic device 102 may transmit the message via one ormore of wireless transmission, wired transmission, device bus, network,etc. For instance, electronic device A 102 a may transmit the message toelectronic device B 102 b. The message may be part of the bitstream 114,for example. In some configurations, electronic device A 102 a may send208 the message to electronic device B 102 b in a separate transmission110 (that is not part of the bitstream 114). For instance, the messagemay be sent using some out-of-band mechanism. In some case, theinformation indicated in 204, 206 may be sent in a SEI message differentthan picture timing SEI message. In yet another case the informationindicated in 204, 206 may be sent in a parameter set e.g. videoparameter set and/or sequence parameter set and/or picture parameter setand/or adaptation parameter set and/or slice header.

FIG. 3 is a flow diagram illustrating one configuration of a method 300for determining one or more removal delays for decoding units in anaccess unit. In other words, the method 300 illustrated in FIG. 3 mayfurther illustrate step 204 in the method 200 illustrated in FIG. 2. Themethod 300 may be performed by an encoder 104. The encoder 104 maydetermine 302 whether to include a common decoding unit CPB removaldelay parameter (e.g., common_du_cpb_removal_delay). This may includedetermining whether a common decoding unit CPB removal delay flag (e.g.,common_du_cpb_removal_delay_flag) is set. An encoder 104 may send thiscommon parameter in case the decoding units are removed from the CPB atregular interval. This may be the case, for example, when each decodingunit corresponds to certain number of rows of the picture or has someother regular structure.

For example, the common decoding unit CPB removal delay flag may be setto 1 when the common decoding unit CPB removal delay parameter is to beincluded in the picture timing SEI message and 0 when it is not to beincluded. If yes (e.g., flag is set to 1), the encoder 104 may determine304 a common decoding unit CPB removal delay parameter (e.g.,common_du_cpb_removal_delay) that is applicable to all decoding units inan access unit. If no (e.g., flag is set to 0), the encoder 104 maydetermine 306 separate decoding unit CPB removal delay parameters (e.g.,du_cpb_removal_delay[i]) for each decoding unit in an access unit.

If a common decoding unit CPB removal delay parameter is present in apicture timing SEI message, it may specify an amount of sub-pictureclock ticks to wait after removal from the CPB 120 of an immediatelypreceding decoding unit before removing from the CPB 120 a currentdecoding unit in the access unit associated with the picture timing SEImessage.

For example, when a decoding unit is a first decoding unit in an accessunit, the common decoding unit CPB 120 removal delay parameter mayspecify an amount of sub-picture clock ticks to wait after removal fromthe CPB 120 of a last decoding unit in an access unit associated with amost recent buffering period SEI message in a preceding access unitbefore removing from the CPB 120 the first decoding unit in the accessunit associated with the picture timing SEI message.

When the decoding unit is a non-first decoding unit in an access unit,the common decoding unit CPB removal delay parameter may specify anamount of sub-picture clock ticks to wait after removal from the CPB 120of a preceding decoding unit in the access unit associated with thepicture timing SEI message before removing from the CPB a currentdecoding unit in the access unit associated with the picture timing SEImessage.

In contrast, when a common decoding unit CPB removal delay parameter(e.g., common_du_cpb_removal delay) is not sent in a picture timing SEImessage, separate decoding unit CPB removal delay parameters (e.g.,du_cpb_removal_delay[i]) may be included in the picture timing SEImessage for each decoding unit in an access unit. The decoding unit CPBremoval delay parameters (e.g., du_cpb_removal_delay[i]) may specify anamount of sub-picture clock ticks to wait after removal from the CPB 120of the last decoding unit before removing from the CPB 120 an i-thdecoding unit in the access unit associated with the picture timing SEImessage. The decoding unit CPB removal delay parameters may becalculated according to a remainder of a modulo 2^((cpb) ^(_) ^(removal)^(_) ^(delay) ^(_) ^(length) ^(_) ^(minus1+1)) counter wherecpb_removal_delay_length_minus1+1 is a length of a common decoding unitCPB removal delay parameter.

FIG. 4 is a flow diagram illustrating one configuration of a method 400for buffering a bitstream. The method 400 may be performed by a decoder112 in an electronic device 102 (e.g., electronic device B 102 b), whichmay receive 402 a message (e.g., a picture timing SEI message or othermessage). For example, the electronic device 102 may receive 402 themessage via one or more of wireless transmission, wired transmission,device bus, network, etc. For instance, electronic device B 102 b mayreceive 402 the message from electronic device A 102 a. The message maybe part of the bitstream 114, for example. In another example,electronic device B 102 b may receive the message from electronic deviceA 102 a in a separate transmission 110 (that is not part of thebitstream 114, for example). For instance, the picture timing SEImessage may be received using some out-of-band mechanism. In someconfigurations, the message may include one or more of a picture timingflag, one or more removal delays for decoding units in an access unitand one or more NAL parameters. Thus, receiving 402 the message mayinclude receiving one or more of a picture timing flag, one or moreremoval delays for decoding units in an access unit and one or more NALparameters.

The decoder 112 may determine 404 whether a CPB 120 operates on anaccess unit level or a sub-picture level. For example, a decoder 112 maydecide to operate on sub-picture basis if it wants to achieve lowlatency. Alternatively, the decision may be based on whether the decoder112 has enough resources to support sub-picture based operation. If theCPB 120 operates on a sub-picture level, the decoder may determine 406one or more removal delays for decoding units in an access unit. Forexample, the decoder 112 may determine a single common decoding unit CPBremoval delay parameter (e.g., common_du_cpb_removal_delay) that isapplicable to all decoding units in the access unit. Alternatively, thedecoder 112 may determine a separate decoding unit CPB removal delay(e.g., du_cpb_removal_delay[i]) for each decoding unit in the accessunit. In other words, the picture timing SEI message may include acommon parameter applicable to all decoding units in an access unit orseparate parameters for every decoding unit.

The decoder 112 may also remove 408 decoding units based on the CPBremoval delays for the decoding units, i.e., using either a commonparameter applicable to all decoding units in an access unit or separateparameters for every decoding unit. The decoder 112 may also decode 410the decoding units.

The decoder 112 may use a variable ClockDiff when determining a removaltime for determined from various signaled parameters. Specifically,ClockDiff may be determined according toClockDiff=(num_units_in_tick−(num_units_in_sub_tick*(num_decoding_units_minus1+1))/time_scale),where num_units_in_tick is number of time units of a clock operating atthe frequency time_scale Hz that corresponds to one increment of a clocktick counter, num_units_in_sub_tick is number of time units of a clockoperating at the frequency time_scale Hz that corresponds to oneincrement of a sub-picture clock tick counter,num_decoding_units_minus1+1 is an amount of decoding units in the accessunit, and time_scale is the number of time units that pass in onesecond.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1, the CPB is operating at sub-picture level andClockDiff is greater than zero, the removal time for decoding unit m,t_(r)(m) is determined according to: t_(r)(m)=t_(r,n)(m)+t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))+ClockDiff wheret_(r,n)(m) is the nominal removal time of the decoding unit m, t_(c)_(_) _(sub) is a sub-picture clock tick, Ceil( ) is a ceiling functionand t_(af)(m) is final arrival time of decoding unit m.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(n)<t_(af)(n), a picture timingflag is set to 1, the CPB is operating at an access unit level andClockDiff is greater than zero, the removal time for access unit n,t_(r)(n) is determined according to:t_(r)(n)=t_(r,n)(n)+t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))−ClockDiff,where t_(r,n)(n) is the nominal removal time of the access unit n,t_(c), is a clock tick, Ceil( ) is a ceiling function and t_(af)(n) is afinal arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the last decoding unit m of access unit, t_(r)(m),according to: t_(r)(m)=t_(r,n)(m)+max((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c)))) where t_(r,n)(m) is thenominal removal time of the last decoding unit m, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,t_(r)(n) according to: t_(r)(n)=t_(r,n)(n)+max((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil(t_(af)(n)−t_(r,n)(n))/t_(c)))) where t_(r,n)(m) is thenominal removal time of the last decoding unit n, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access nit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the last decoding unit m of access unit, t_(r)(m)according to: t_(r)(m)=t_(r,n)(m)+min((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub)),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))) where t_(r,n)(m) is thenominal removal time of the last decoding unit m, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,t_(r)(n) according to: t_(r)(n)=t_(r,n)(n)+min((t_(c) _(_)_(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c) _(_) _(sub))),(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c)))) where t_(r,n)(m) is thenominal removal time of the last decoding unit n, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the last decoding unit m of access unit, t_(r)(m)according to:t_(r)(m)=t_(r,n)(m)+(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit m,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, is clocktick and t_(af)(n) is a final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,t_(r)(n) according to:t_(r)(n)=t_(r,n)(n)+(t_(c)*Ceil((t_(af)(n)−t_(r,n)(n))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit n,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick and t_(af)(n) is a final arrival time of access unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m) <t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the decoding unit which is not the last decoding unitof the access unit is set as t_(r)(m)=t_(af)(m), where t_(af)(m) is afinal arrival time of decoding unit m. And the removal time for the lastdecoding unit m of access unit, t_(r)(m) according to:t_(r)(m)=t_(r,n)(m)+(t_(c) _(_) _(sub)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c)_(_) _(sub))) where t_(r,n)(m) is the nominal removal time of the lastdecoding unit m, t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) isa ceiling function, t_(af)(m) is a final arrival time of last decodingunit m, t_(r,n)(n) is the nominal removal time of the access unit n, tois clock tick, t_(af)(n) is a final arrival time of access unit n andt_(af)(m) is a final arrival time of the last decoding unit in theaccess unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the decoding unit which is not the last decoding unitof the access unit is set as t_(r)(m)=t_(af)(m), where t_(af)(m) is afinal arrival time of decoding unit m. And the removal time for the lastdecoding unit m of access unit, t_(r)(m) according to:t_(r)(m)=t_(r,n)(m)+(t_(c)*Ceil((t_(af)(m)−t_(r,n)(m))/t_(c))) wheret_(r,n)(m) is the nominal removal time of the last decoding unit m,t_(c) _(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceilingfunction, t_(af)(m) is a final arrival time of last decoding unit m,t_(r,n)(n) is the nominal removal time of the access unit n, t_(c) isclock tick, t_(af)(n) is a final arrival time of access unit n andt_(af)(m) is a final arrival time of the last decoding unit in theaccess unit n.

When a low delay hypothetical reference decoder (HRD) flag (e.g.,low_delay_hrd_flag) is set to 1, t_(r,n)(m)<t_(af)(m), a picture timingflag is set to 1 and the CPB is operating at sub-picture level, theremoval time for the decoding unit is set as t_(r)(m)=t_(af)(m) wheret_(r,n)(m) is the nominal removal time of the decoding unit m, t_(c)_(_) _(sub) is sub-picture clock tick, Ceil( ) is a ceiling function,t_(af)(m) is a final arrival time of decoding unit m, t_(r,n)(n) is thenominal removal time of the access unit n, t_(c) is clock tick,t_(af)(n) is a final arrival time of access unit n and t_(af)(m) is afinal arrival time of the decoding unit in the access unit n.

When a low delay hypothetical reference decoder (HRD) flag is set to 1,t_(r,n)(n)<t_(af)(n), a picture timing flag is set to 1 and the CPB isoperating at access unit level, the removal time for access unit n,t_(r)(n) according to: t_(r)(n)=t_(af)(n) where t_(r,n)(m) is thenominal removal time of the last decoding unit n, t_(c) _(_) _(sub) issub-picture clock tick, Ceil( ) is a ceiling function, t_(af)(m) is afinal arrival time of last decoding unit m, t_(r,n)(n) is the nominalremoval time of the access unit n, t_(c) is clock tick and t_(af)(n) isa final arrival time of access unit n.

If the CPB operates on an access unit level, the decoder 112 maydetermine 412 a CPB removal delay parameter. This may be included in thereceived picture timing SEI message (e.g., cpb_removal_delay). Thedecoder 112 may also remove 414 an access unit based on the CPB removaldelay parameter and decode 416 the access unit. In other words, thedecoder 112 may decode whole access units at a time, rather thandecoding units within the access unit.

FIG. 5 is a flow diagram illustrating one configuration of a method 500for determining one or more removal delays for decoding units in anaccess unit. In other words, the method 500 illustrated in FIG. 5 mayfurther illustrate step 406 in the method 400 illustrated in FIG. 4. Themethod 500 may be performed by a decoder 112. The decoder 112 maydetermine 502 whether a received picture timing SEI message includes acommon decoding unit CPB removal delay parameter. This may includedetermining whether a common decoding unit CPB removal delay flag (e.g.,common_du_cpb_removal_delay_flag) is set. If yes, the decoder 112 maydetermine 504 a common decoding unit CPB removal delay parameter (e.g.,common_du_cpb_removal_delay) that is applicable to all decoding units inan access unit. If no, the decoder 112 may determine 506 separatedecoding unit CPB removal delay parameters (e.g.,du_cpb_removal_delay[i]) for each decoding unit in an access unit.

In addition to modifying the picture timing SEI message semantics, thepresent systems and methods may also impose a bitstream constraint sothat the operation of sub-picture based CPB operation and access unitbased CPB operation result in the same timing of decoding unit removal.Specifically, when the picture timing flag (e.g.,sub_pic_cpb_params_present_fiag) is set to 1, the CPB removal delay maybe set according to

$\begin{matrix}{{{cpb\_ removal}{\_ delay}} = \frac{\left( {\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_ minus}\; 1}{{du\_ cpb}{\_ removal}{{\_ delay}\lbrack i\rbrack}*t_{c,{sub}}}} \right.}{t_{c}}} & \left\lbrack {{Math}.\mspace{14mu} 6} \right\rbrack\end{matrix}$

where du_cpb_removal_delay[i] are the decoding unit CPB removal delayparameters, t_(c) is a clock tick, t_(c) _(_) _(sub) is a sub-pictureclock tick, num_decoding_units_minus1 is an amount of decoding units inthe access unit offset by one, and i is an index.

Alternatively, the CPB removal delay may be set as described next: Letthe variable Tdu(k) be defined as:

$\begin{matrix}{{T_{du}(k)} = {{T_{du}\left( {k - 1} \right)} + {t_{c,{sub}}*{\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_ minus}\; 1_{k}}\left( {{{du\_ cpb}{\_ removal}{\_ delay}{\_ minus}\;{1_{k}\lbrack i\rbrack}} + 1} \right)}}}} & \left\lbrack {{Math}.\mspace{14mu} 7} \right\rbrack\end{matrix}$

where du_cpb_removal_delay_minus1_(k)[i] andnum_decoding_units_minus1_(k) are parameters for i′th decoding unit ofk′th access unit (with k=0 for the access unit that initialized the HRDand T_(du)(k)=0 for k<1), and wheredu_cpb_removal_delay_minus1_(k)[i]+1=du_cpb_removal delay_(k)[i] is thedecoding unit CPB removal delay parameter for the I′th decoding unit ofthe k′th access unit, and num_decoding_units_minus1_(k) is the number ofdecoding units in the k′th access unit, t_(c) is a clock tick, t_(c,sub)is a sub-picture clock tick, and i and k are an indices. Then when thepicture timing flag (e.g., sub_pic_cpb_params_present_flag) is set to 1,the following condition shall be true:(au_cpb_removal_delay_minus1+1)*t_(c)==T_(du)(k), where(au_cpb_removal_delay_minus1+1)=cpb_removal_delay, the CPB removaldelay. Thus in this case the CPB removal delay(au_cpb_removal_delay_minus1+1) is set such that the operation ofsub-picture based CPB operation and access unit based CPB operationresult in the same timing of access unit removal and last decoding unitof the access unit removal.

Alternatively, the CPB removal delay may be set according to

$\begin{matrix}{{- 1} \leq \left\lbrack {{{cpb\_ removal}{\_ delay}*t_{c}} - \left( {\sum\limits_{i = 0}^{{num\_ decoding}{\_ units}{\_ minus}\; 1}{{du\_ cpb}{\_ removal}{{\_ delay}\lbrack i\rbrack}*t_{c,{sub}}}} \right)} \right\rbrack \leq 1} & \left\lbrack {{Math}.\mspace{14mu} 8} \right\rbrack\end{matrix}$where du_cpb_removal_delay[i] are the decoding unit CPB removal delayparameters, t_(c) is a clock tick, t_(c) _(_) _(sub) is a sub-pictureclock tick, num_decoding_units_minus1 is an amount of decoding units inthe access unit offset by one and i is an index.

Alternatively, cpb_removal_delay anddu_cpb_removal_delay[num_decoding_units_minus1] may be set according to:cpb_removal_delay*t_(c)=du_cpb_removal_delay[num_decoding_units_minus1]*t_(c,sub)where du_cpb_removal_delay[num_decoding_units_minus1] is the decodingunit CPB removal delay parameter for the num_decoding_units_minus1′thdecoding unit, t_(c) is a clock tick, t_(c,sub) is a sub-picture clocktick, num_decoding_units_minus1 is an amount of decoding units in theaccess unit offset by one.

In addition to modifying the picture timing SEI message semantics, thepresent systems and methods may also impose a bitstream constraint sothat the operation of sub-picture based CPB operation and access unitbased CPB operation result in the same timing of decoding unit removal.Specifically, when the picture timing flag (e.g.,sub_pic_cpb_params_present_flag) is set to 1, the values forcpb_removal_delay and du_cpb_removal_delay[num_decoding_units minus1]may be set so as to satisfy:−1<=(cpb_removal_delay*t_(c)−du_cpb_removal_delay[num_decoding_units_minus1]*t_(c,sub))<=−1where du_cpb_removal_delay[num_decoding_units_minus1] is the decodingunit CPB removal delay parameter for the num_decoding_units_minus1′thdecoding unit, t_(c) is a clock tick, t_(c,sub) is a sub-picture clocktick num_decoding_units_minus1 is an amount of decoding units in theaccess unit offset by one.

FIG. 6 is a block diagram illustrating one configuration of an encoder604 on an electronic device 602. It should be noted that one or more ofthe elements illustrated as included within the electronic device 602may be implemented in hardware, software or a combination of both. Forexample, the electronic device 602 includes an encoder 604, which may beimplemented in hardware, software or a combination of both. Forinstance, the encoder 604 may be implemented as a circuit, integratedcircuit, application-specific integrated circuit (ASIC), processor inelectronic communication with memory with executable instructions,firmware, field-programmable gate array (FPGA), etc., or a combinationthereof. In some configurations, the encoder 604 may be a HEVC coder.

The electronic device 602 may include a source 622. The source 622 mayprovide picture or image data (e.g., video) as one or more inputpictures 606 to the encoder 604. Examples of the source 622 may includeimage sensors, memory, communication interfaces, network interfaces,wireless receivers, ports, etc.

One or more input pictures 606 may be provided to an intra-frameprediction module and reconstruction buffer 624. An input picture 606may also be provided to a motion estimation and motion compensationmodule 646 and to a subtraction module 628.

The intra-frame prediction module and reconstruction buffer 624 maygenerate intra mode information 640 and an intra-signal 626 based on oneor more input pictures 606 and reconstructed data 660. The motionestimation and motion compensation module 646 may generate inter modeinformation 648 and an inter signal 644 based on one or more inputpictures 606 and a reference picture buffer 676 signal 678. In someconfigurations, the reference picture buffer 676 may include data fromone or more reference pictures in the reference picture buffer 676.

The encoder 604 may select between the intra signal 626 and the intersignal 644 in accordance with a mode. The intra signal 626 may be usedin order to exploit spatial characteristics within a picture in anintra-coding mode. The inter signal 644 may be used in order to exploittemporal characteristics between pictures in an inter coding mode. Whilein the intra coding mode, the intra signal 626 may be provided to thesubtraction module 628 and the intra mode information 640 may beprovided to an entropy coding module 642. While in the inter codingmode, the inter signal 644 may be provided to the subtraction module 628and the inter mode information 648 may be provided to the entropy codingmodule 642.

Either the intra signal 626 or the inter signal 644 (depending on themode) is subtracted from an input picture 606 at the subtraction module628 in order to produce a prediction residual 630. The predictionresidual 630 is provided to a transformation module 632. Thetransformation module 632 may compress the prediction residual 630 toproduce a transformed signal 634 that is provided to a quantizationmodule 636. The quantization module 636 quantizes the transformed signal634 to produce transformed and quantized coefficients (TQCs) 638.

The TQCs 638 are provided to an entropy coding module 642 and an inversequantization module 650. The inverse quantization module 650 performsinverse quantization on the TQCs 638 to produce an inverse quantizedsignal 652 that is provided to an inverse transformation module 654. Theinverse transformation module 654 decompresses the inverse quantizedsignal 652 to produce a decompressed signal 656 that is provided to areconstruction module 658.

The reconstruction module 658 may produce reconstructed data 660 basedon the decompressed signal 656. For example, the reconstruction module658 may reconstruct (modified) pictures. The reconstructed data 660 maybe provided to a deblocking filter 662 and to the intra predictionmodule and reconstruction buffer 624. The deblocking filter 662 mayproduce a filtered signal 664 based on the reconstructed data 660.

The filtered signal 664 may be provided to a sample adaptive offset(SAO) module 666. The SAO module 666 may produce SAO information 668that is provided to the entropy coding module 642 and an SAO signal 670that is provided to an adaptive loop filter (ALF) 672. The ALF 672produces an ALF signal 674 that is provided to the reference picturebuffer 676. The ALF signal 674 may include data from one or morepictures that may be used as reference pictures.

The entropy coding module 642 may code the TQCs 638 to produce bitstreamA 614 a (e.g., encoded picture data). For example, the entropy codingmodule 642 may code the TQCs 638 using Context-Adaptive Variable LengthCoding (CAVLC) or Context-Adaptive Binary Arithmetic Coding (CABAC). Inparticular, the entropy coding module 642 may code the TQCs 638 based onone or more of intra mode information 640, inter mode information 648and SAO information 668. Bitstream A 614 a (e.g., encoded picture data)may be provided to a message generation module 608. The messagegeneration module 608 may be configured similarly to the messagegeneration module 108 described in connection with FIG. 1. Additionallyor alternatively, the message generation module 608 may perform one ormore of the procedures described in connection with FIG. 2 and FIG. 3.

For example, the message generation module 608 may generate a message(e.g., picture timing SEI message or other message) includingsub-picture parameters. The sub-picture parameters may include one ormore removal delays for decoding units (e.g.,common_du_cpb_removal_delay or du_cpb_removal_delay[i]) and one or moreNAL parameters (e.g., common_num_nalus_in_du_minus1 ornum_nalus_in_du_minus1[i]). In some configurations, the message may beinserted into bitstream A 614 a to produce bitstream B 614 b. Thus, themessage may be generated after the entire bitstream A 614 a is generated(e.g., after most of bitstream B 614 b is generated), for example. Inother configurations, the message may not be inserted into bitstream A614 a (in which case bitstream B 614 b may be the same as bitstream A614 a), but may be provided in a separate transmission 610.

In some configurations, the electronic device 602 sends the bitstream614 to another electronic device. For example, the bitstream 614 may beprovided to a communication interface, network interface, wirelesstransmitter, port, etc. For instance, the bitstream 614 may betransmitted to another electronic device via LAN, the Internet, acellular phone base station, etc. The bitstream 614 may additionally oralternatively be stored in memory or other component on the electronicdevice 602.

FIG. 7 is a block diagram illustrating one configuration of a decoder712 on an electronic device 702. The decoder 712 may be included in anelectronic device 702. For example, the decoder 712 may be a HEVCdecoder. The decoder 712 and one or more of the elements illustrated asincluded in the decoder 712 may be implemented in hardware, software ora combination of both. The decoder 712 may receive a bitstream 714(e.g., one or more encoded pictures and overhead data included in thebitstream 714) for decoding. In some configurations, the receivedbitstream 714 may include received overhead data, such as a message(e.g., picture timing SEI message or other message), slice header, PPS,etc. In some configurations, the decoder 712 may additionally receive aseparate transmission 710. The separate transmission 710 may include amessage (e.g., a picture timing SEI message or other message). Forexample, a picture timing SEI message or other message may be receivedin a separate transmission 710 instead of in the bitstream 714. However,it should be noted that the separate transmission 710 may be optionaland may not be utilized in some configurations.

The decoder 712 includes a CPB 720. The CPB 720 may be configuredsimilarly to the CPB 120 described in connection with FIG. 1 above.Additionally or alternatively, the decoder 712 may perform one or moreof the procedures described in connection with FIG. 4 and FIG. 5. Forexample, the decoder 712 may receive a message (e.g., picture timing SEImessage or other message) with sub-picture parameters and remove anddecode decoding units in an access unit based on the sub-pictureparameters. It should be noted that one or more access units may beincluded in the bitstream and may include one or more of encoded picturedata and overhead data.

The Coded Picture Buffer (CPB) 720 may provide encoded picture data toan entropy decoding module 701. The encoded picture data may be entropydecoded by an entropy decoding module 701, thereby producing a motioninformation signal 703 and quantized, scaled and/or transformedcoefficients 705.

The motion information signal 703 may be combined with a portion of areference frame signal 798 from a frame memory 709 at a motioncompensation module 780, which may produce an inter-frame predictionsignal 782. The quantized, descaled and/or transformed coefficients 705may be inverse quantized, scaled and inverse transformed by an inversemodule 707, thereby producing a decoded residual signal 784. The decodedresidual signal 784 may be added to a prediction signal 792 to produce acombined signal 786. The prediction signal 792 may be a signal selectedfrom either the inter-frame prediction signal 782 produced by the motioncompensation module 780 or an intra-frame prediction signal 790 producedby an intra-frame prediction module 788. In some configurations, thissignal selection may be based on (e.g., controlled by) the bitstream714.

The intra-frame prediction signal 790 may be predicted from previouslydecoded information from the combined signal 786 (in the current frame,for example). The combined signal 786 may also be filtered by ade-blocking filter 794. The resulting filtered signal 796 may be writtento frame memory 709. The resulting filtered signal 796 may include adecoded picture. The frame memory 709 may provide a decoded picture 718.In some cases 709 may be a decoded picture buffer.

FIG. 8 illustrates various components that may be utilized in atransmitting electronic device 802. One or more of the electronicdevices 102, 602, 702 described herein may be implemented in accordancewith the transmitting electronic device 802 illustrated in FIG. 8.

The transmitting electronic device 802 includes a processor 817 thatcontrols operation of the electronic device 802. The processor 817 mayalso be referred to as a CPU. Memory 811, which may include bothread-only memory (ROM), random access memory (RAM) or any type of devicethat may store information, provides instructions 813 a (e.g.,executable instructions) and data 815 a to the processor 817. A portionof the memory 811 may also include non-volatile random access memory(NVRAM). The memory 811 may be in electronic communication with theprocessor 817.

Instructions 813 b and data 815 b may also reside in the processor 817.Instructions 813 b and/or data 815 b loaded into the processor 817 mayalso include instructions 813 a and/or data 815 a from memory 811 thatwere loaded for execution or processing by the processor 817. Theinstructions 813 b may be executed by the processor 817 to implement thesystems and methods disclosed herein. For example, the instructions 813b may be executable to perform one or more of the methods 200, 300, 400,500 described above.

The transmitting electronic device 802 may include one or morecommunication interfaces 819 for communicating with other electronicdevices (e.g., receiving electronic device). The communicationinterfaces 819 may be based on wired communication technology, wirelesscommunication technology, or both. Examples of a communication interface819 include a serial port, a parallel port, a Universal Serial Bus(USB), an Ethernet adapter, an IEEE 1394 bus interface, a small computersystem interface (SCSI) bus interface, an infrared (IR) communicationport, a Bluetooth wireless communication adapter, a wireless transceiverin accordance with 3^(rd) Generation Partnership Project (3GPP)specifications and so forth.

The transmitting electronic device 802 may include one or more outputdevices 823 and one or more input devices 821. Examples of outputdevices 823 include a speaker, printer, etc. One type of output devicethat may be included in an electronic device 802 is a display device825. Display devices 825 used with configurations disclosed herein mayutilize any suitable image projection technology, such as a cathode raytube (CRT), liquid crystal display (LCD), light-emitting diode (LED),gas plasma, electroluminescence or the like. A display controller 827may be provided for converting data stored in the memory 811 into text,graphics, and/or moving images (as appropriate) shown on the display825. Examples of input devices 821 include a keyboard, mouse,microphone, remote control device, button, joystick, trackball,touchpad, touchscreen, lightpen, etc.

The various components of the transmitting electronic device 802 arecoupled together by a bus system 829, which may include a power bus, acontrol signal bus and a status signal bus, in addition to a data bus.However, for the sake of clarity, the various buses are illustrated inFIG. 8 as the bus system 829. The transmitting electronic device 802illustrated in FIG. 8 is a functional block diagram rather than alisting of specific components.

FIG. 9 is a block diagram illustrating various components that may beutilized in a receiving electronic device 902. One or more of theelectronic devices 102, 602, 702 described herein may be implemented inaccordance with the receiving electronic device 902 illustrated in FIG.9.

The receiving electronic device 902 includes a processor 917 thatcontrols operation of the electronic device 902. The processor 917 mayalso be referred to as a CPU. Memory 911, which may include bothread-only memory (ROM), random access memory (RAM) or any type of devicethat may store information, provides instructions 913 a (e.g.,executable instructions) and data 915 a to the processor 917. A portionof the memory 911 may also include non-volatile random access memory(NVRAM). The memory 911 may be in electronic communication with theprocessor 917.

Instructions 913 b and data 915 b may also reside in the processor 917.Instructions 913 b and/or data 915 b loaded into the processor 917 mayalso include instructions 913 a and/or data 915 a from memory 911 thatwere loaded for execution or processing by the processor 917. Theinstructions 913 b may be executed by the processor 917 to implement thesystems and methods disclosed herein. For example, the instructions 913b may be executable to perform one or more of the methods 200, 300, 400,500 described above.

The receiving electronic device 902 may include one or morecommunication interfaces 919 for communicating with other electronicdevices (e.g., a transmitting electronic device). The communicationinterface 919 may be based on wired communication technology, wirelesscommunication technology, or both. Examples of a communication interface919 include a serial port, a parallel port, a Universal Serial Bus(USB), an Ethernet adapter, an IEEE 1394 bus interface, a small computersystem interface (SCSI) bus interface, an infrared (IR) communicationport, a Bluetooth wireless communication adapter, a wireless transceiverin accordance with 3^(rd) Generation Partnership Project (3GPP)specifications and so forth.

The receiving electronic device 902 may include one or more outputdevices 923 and one or more input devices 921. Examples of outputdevices 923 include a speaker, printer, etc. One type of output devicethat may be included in an electronic device 902 is a display device925. Display devices 925 used with configurations disclosed herein mayutilize any suitable image projection technology, such as a cathode raytube (CRT), liquid crystal display (LCD), light-emitting diode (LED),gas plasma, electroluminescence or the like. A display controller 927may be provided for converting data stored in the memory 911 into text,graphics, and/or moving images (as appropriate) shown on the display925. Examples of input devices 921 include a keyboard, mouse,microphone, remote control device, button, joystick, trackball,touchpad, touchscreen, lightpen, etc.

The various components of the receiving electronic device 902 arecoupled together by a bus system 929, which may include a power bus, acontrol signal bus and a status signal bus, in addition to a data bus.However, for the sake of clarity, the various buses are illustrated inFIG. 9 as the bus system 929. The receiving electronic device 902illustrated in FIG. 9 is a functional block diagram rather than alisting of specific components.

FIG. 10 is a block diagram illustrating one configuration of anelectronic device 1002 in which systems and methods for sending amessage may be implemented. The electronic device 1002 includes encodingmeans 1031 and transmitting means 1033. The encoding means 1031 andtransmitting means 1033 may be configured to perform one or more of thefunctions described in connection with one or more of FIG. 1, FIG. 2,FIG. 3, FIG. 6 and FIG. 8 above. For example, the encoding means 1031and transmitting means 1033 may generate a bitstream 1014. FIG. 8 aboveillustrates one example of a concrete apparatus structure of FIG. 10.Other various structures may be implemented to realize one or more ofthe functions of FIG. 1, FIG. 2, FIG. 3, FIG. 6 and FIG. 8. For example,a DSP may be realized by software.

FIG. 11 is a block diagram illustrating one configuration of anelectronic device 1102 in which systems and methods for buffering abitstream 1114 may be implemented. The electronic device 1102 mayinclude receiving means 1135 and decoding means 1137. The receivingmeans 1135 and decoding means 1137 may be configured to perform one ormore of the functions described in connection with one or more of FIG.1, FIG. 4, FIG. 5, FIG. 7 and FIG. 9 above. For example, the receivingmeans 1135 and decoding means 1137 may receive a bitstream 1114. FIG. 9above illustrates one example of a concrete apparatus structure of FIG.11. Other various structures may be implemented to realize one or morefunctions of FIG. 1, FIG. 4, FIG. 5, FIG. 7 and FIG. 9. For example, aDSP may be realized by software.

The term “computer-readable medium” refers to any available medium thatcan be accessed by a computer or a processor. The term“computer-readable medium,” as used herein, may denote a computer-and/or processor-readable medium that is non-transitory and tangible. Byway of example, and not limitation, a computer-readable orprocessor-readable medium may comprise RAM, ROM, EEPROM, CD-ROM or otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium that can be used to carry or store desiredprogram code in the form of instructions or data structures and that canbe accessed by a computer or processor. Disk and disc, as used herein,includes compact disc (CD), laser disc, optical disc, digital versatiledisc (DVD), floppy disk and Blu-ray (registered trademark) disc wheredisks usually reproduce data magnetically, while discs reproduce dataoptically with lasers.

It should be noted that one or more of the methods described herein maybe implemented in and/or performed using hardware. For example, one ormore of the methods or approaches described herein may be implemented inand/or realized using a chipset, an ASIC, a large-scale integratedcircuit (LSI) or integrated circuit, etc.

Each of the methods disclosed herein comprises one or more steps oractions for achieving the described method. The method steps and/oractions may be interchanged with one another and/or combined into asingle step without departing from the scope of the claims. In otherwords, unless a specific order of steps or actions is required forproper operation of the method that is being described, the order and/oruse of specific steps and/or actions may be modified without departingfrom the scope of the claims.

It is to be understood that the claims are not limited to the preciseconfiguration and components illustrated above. Various modifications,changes and variations may be made in the arrangement, operation anddetails of the systems, methods, and apparatus described herein withoutdeparting from the scope of the claims.

The invention claimed is:
 1. A method for decoding a video bitstreamcorresponding to a video sequence comprising: (a) receiving said videobitstream; (b) receiving a picture timing SEI message in said videobitstream; (c) receiving information indicates the number of decodingunits in an access unit, associated with said picture timing SEImessage; (d) receiving a common decoding unit Coded Picture Buffer (CPB)removal delay flag in said picture timing SEI message; (e) when saidcommon decoding unit CPB removal delay flag is true, receiving a commondecoding unit CPB removal delay; (f) when said common decoding unit CPBremoval delay flag is false, receiving a decoding unit CPB removal delayfor each of decoding units of the access unit associated with thepicture timing SEI message; (g) decoding said video sequence from saidvideo bitstream by using said common decoding unit CPB removal delayflag, said common decoding unit CPB removal delay and said decoding unitCPB removal delay for each of decoding units of the access unit.
 2. Themethod of claim 1 wherein the common decoding unit CPB removal delay isthe number of sub-picture clock ticks to wait after removal from the CPBof the preceding decoding unit before removing from the CPB the currentdecoding unit in the access unit associated with the picture timing SEImessage.
 3. The method of claim 1 wherein the decoding unit CPB removaldelay for the i-th decoding unit, is the number sub-picture clock ticksto wait after removal from the CPB of the first decoding unit in theaccess unit associated with the most recent buffering period SEI messagein a preceding access unit, in the access unit associated with thepicture timing SEI message, before removing from the CPB the i-thdecoding unit in the access unit associated with the picture timing SEImessage.
 4. A decoder for decoding a video bitstream corresponding to avideo sequence, the decoder, comprising: a memory and a processor,wherein the processor configured to perform steps of: (a) receiving saidvideo bitstream; (b) receiving a picture timing SEI message in saidvideo bitstream; (c) receiving a number of decoding units in an accessunit, associated with said picture timing SEI message; (d) receiving acommon decoding unit Coded Picture Buffer (CPB) removal delay flag insaid picture timing SEI message; (e) when said common decoding unit CPBremoval delay flag is true, receiving a common decoding unit CPB removaldelay; (f) when said common decoding unit CPB removal delay flag isfalse, receiving a decoding unit CPB removal delay for each of decodingunits of the access unit associated with the picture timing SEI message;(g) decoding said video sequence from said video bitstream by using saidcommon decoding unit CPB removal delay flag, common decoding unit CPBremoval delay and decoding unit CPB removal delay for each of decodingunits of the access unit.
 5. The decoder of claim 4 wherein the commondecoding unit CPB removal delay specifies the number of sub-pictureclock ticks to wait after removal from the CPB of the preceding decodingunit before removing from the CPB the current decoding unit in theaccess unit associated with the picture timing SEI message.
 6. Thedecoder of claim 1 wherein the decoding unit CPB removal delay for thei-th decoding unit, in the access unit associated with picture timingSEI message, is the number sub-picture clock ticks to wait after removalfrom the CPB of the first decoding unit in the access unit associatedwith the most recent buffering period SEI message in a preceding accessunit, in the access unit associated with the picture timing SEI message,before removing from the CPB the i-th decoding unit in the access unitassociated with the picture timing SEI message.